# Testing DLA using object detection model and deepstream

**URL:** <https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667>\
**Category:** DeepStream SDK\
**Tags:** dla\
**Created:** [August 8, 2024, 2:44am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667 "2024-08-08T02:44:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rishikesan](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rishikesan](https://forums.developer.nvidia.com/u/rishikesan)\
**Post date:** [August 8, 2024, 2:44am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/1 "2024-08-08T02:44:36Z")

</div>

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)**  
**• DeepStream Version**  
**• JetPack Version (valid for Jetson only)**  
**• TensorRT Version**  
**• NVIDIA GPU Driver Version (valid for GPU only)**  
\*\*• Issue Type( questions,)

I am testing the DLA functionality to make use of its power in the edge devices

From the below reference

> **[GitHub - NVIDIA-AI-IOT/jetson\_dla\_tutorial: A tutorial for getting started with the Deep...](https://github.com/NVIDIA-AI-IOT/jetson_dla_tutorial)**
>
> A tutorial for getting started with the Deep Learning Accelerator (DLA) on NVIDIA Jetson

We understood that the DLA should increase the throughput while running the inference

We tested by taking ONNX version of yolov8s and run the trtexec to generate the engine file with below 2 ways to check the difference between model running with DLA and without

```auto
trtexec --onnx=onnx_models/yolov8s.onnx --saveEngine=yolov8s.engine --exportProfile=model_gn.json --int8 --useDLACore=0 --allowGPUFallback --useSpinWait --separateProfileRun > model_gn.log

trtexec --onnx=onnx_models/yolov8s.onnx --saveEngine=yolov8s.engine --int8 > model_gn.log

```

While it is running as we checked the throughput as below  
Engine with DLA throughput : ~ 140 qps  
Engine without DLA throughput : ~ 201 qps  
As we understood this should be other way around if DLA performs better on the resources

Below i have attached the generated profile for the model with DLA for the reference

```auto
[
  { "count" : 433 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to {ForeignNode[/0/model.0/conv/Conv.../0/model.22/Concat_2]}", "timeMs" : 47.0022, "averageMs" : 0.10855, "medianMs" : 0.108544, "percentage" : 1.5176 }
, { "name" : "{ForeignNode[/0/model.0/conv/Conv.../0/model.22/Concat_2]}", "timeMs" : 2543.14, "averageMs" : 5.87329, "medianMs" : 5.86982, "percentage" : 82.1123 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to /0/model.22/Reshape", "timeMs" : 37.5533, "averageMs" : 0.0867282, "medianMs" : 0.086688, "percentage" : 1.21251 }
, { "name" : "/0/model.22/Reshape", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_copy_output", "timeMs" : 31.9359, "averageMs" : 0.073755, "medianMs" : 0.073696, "percentage" : 1.03114 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to /0/model.22/Reshape_1", "timeMs" : 13.1777, "averageMs" : 0.0304335, "medianMs" : 0.030432, "percentage" : 0.42548 }
, { "name" : "/0/model.22/Reshape_1", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_1_copy_output", "timeMs" : 9.47417, "averageMs" : 0.0218803, "medianMs" : 0.021888, "percentage" : 0.3059 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to /0/model.22/Reshape_2", "timeMs" : 6.68272, "averageMs" : 0.0154335, "medianMs" : 0.015456, "percentage" : 0.21577 }
, { "name" : "/0/model.22/Reshape_2", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_2_copy_output", "timeMs" : 3.87344, "averageMs" : 0.00894559, "medianMs" : 0.009024, "percentage" : 0.125065 }
, { "name" : "/0/model.22/dfl/Reshape + /0/model.22/dfl/Transpose", "timeMs" : 37.558, "averageMs" : 0.0867391, "medianMs" : 0.08672, "percentage" : 1.21267 }
, { "name" : "/0/model.22/dfl/Softmax", "timeMs" : 30.5957, "averageMs" : 0.0706599, "medianMs" : 0.070624, "percentage" : 0.987869 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to /0/model.22/dfl/Softmax", "timeMs" : 14.4346, "averageMs" : 0.0333362, "medianMs" : 0.033312, "percentage" : 0.466061 }
, { "name" : "/0/model.22/dfl/conv/Conv", "timeMs" : 20.9888, "averageMs" : 0.0484729, "medianMs" : 0.04848, "percentage" : 0.677681 }
, { "name" : "/0/model.22/Constant_3_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_3_output_0_clone_1", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_3_output_0_clone_2", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_9_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_10_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 240) [Constant] + (Unnamed Layer* 241) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_16_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_17_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 254) [Constant] + (Unnamed Layer* 255) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_23_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Constant_24_output_0", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 268) [Constant] + (Unnamed Layer* 269) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Expand", "timeMs" : 3.42969, "averageMs" : 0.00792077, "medianMs" : 0.008, "percentage" : 0.110737 }
, { "name" : "/0/model.22/Expand_1", "timeMs" : 2.66739, "averageMs" : 0.00616026, "medianMs" : 0.006144, "percentage" : 0.0861242 }
, { "name" : "/0/model.22/Expand_2", "timeMs" : 2.8393, "averageMs" : 0.00655727, "medianMs" : 0.006624, "percentage" : 0.0916747 }
, { "name" : "/0/model.22/Expand_3", "timeMs" : 2.65347, "averageMs" : 0.00612811, "medianMs" : 0.006144, "percentage" : 0.0856748 }
, { "name" : "/0/model.22/Expand_4", "timeMs" : 2.83373, "averageMs" : 0.00654441, "medianMs" : 0.006688, "percentage" : 0.0914949 }
, { "name" : "/0/model.22/Expand_5", "timeMs" : 2.54567, "averageMs" : 0.00587914, "medianMs" : 0.006016, "percentage" : 0.082194 }
, { "name" : "/0/model.22/Unsqueeze_1", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Unsqueeze_1_copy_output", "timeMs" : 3.32502, "averageMs" : 0.00767904, "medianMs" : 0.007712, "percentage" : 0.107358 }
, { "name" : "/0/model.22/Unsqueeze", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 242) [Slice]", "timeMs" : 2.92838, "averageMs" : 0.00676301, "medianMs" : 0.006784, "percentage" : 0.0945511 }
, { "name" : "/0/model.22/Unsqueeze_3", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Unsqueeze_3_copy_output", "timeMs" : 2.76176, "averageMs" : 0.00637819, "medianMs" : 0.006336, "percentage" : 0.0891711 }
, { "name" : "/0/model.22/Unsqueeze_2", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 256) [Slice]", "timeMs" : 2.55299, "averageMs" : 0.00589606, "medianMs" : 0.005856, "percentage" : 0.0824306 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to (Unnamed Layer* 256) [Slice]", "timeMs" : 3.14115, "averageMs" : 0.00725439, "medianMs" : 0.007264, "percentage" : 0.101421 }
, { "name" : "/0/model.22/Unsqueeze_5", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Unsqueeze_5_copy_output", "timeMs" : 2.80368, "averageMs" : 0.00647502, "medianMs" : 0.0064, "percentage" : 0.0905248 }
, { "name" : "/0/model.22/Unsqueeze_4", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "(Unnamed Layer* 270) [Slice]", "timeMs" : 2.53015, "averageMs" : 0.00584329, "medianMs" : 0.005824, "percentage" : 0.0816929 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to (Unnamed Layer* 270) [Slice]", "timeMs" : 2.66938, "averageMs" : 0.00616484, "medianMs" : 0.006144, "percentage" : 0.0861883 }
, { "name" : "/0/model.22/Squeeze + (Unnamed Layer* 243) [Shuffle]_copy_input", "timeMs" : 2.69658, "averageMs" : 0.00622766, "medianMs" : 0.006208, "percentage" : 0.0870665 }
, { "name" : "/0/model.22/Squeeze + (Unnamed Layer* 243) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Squeeze_1 + (Unnamed Layer* 257) [Shuffle]_copy_input", "timeMs" : 2.44269, "averageMs" : 0.00564132, "medianMs" : 0.0056, "percentage" : 0.0788691 }
, { "name" : "/0/model.22/Squeeze_1 + (Unnamed Layer* 257) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Squeeze_2 + (Unnamed Layer* 271) [Shuffle]_copy_input", "timeMs" : 2.45511, "averageMs" : 0.00566999, "medianMs" : 0.005632, "percentage" : 0.07927 }
, { "name" : "/0/model.22/Squeeze_2 + (Unnamed Layer* 271) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Unsqueeze_output_0 copy", "timeMs" : 3.092, "averageMs" : 0.00714088, "medianMs" : 0.0072, "percentage" : 0.0998339 }
, { "name" : "/0/model.22/Unsqueeze_2_output_0 copy", "timeMs" : 2.79174, "averageMs" : 0.00644745, "medianMs" : 0.006368, "percentage" : 0.0901393 }
, { "name" : "/0/model.22/Unsqueeze_4_output_0 copy", "timeMs" : 2.75017, "averageMs" : 0.00635144, "medianMs" : 0.006272, "percentage" : 0.0887971 }
, { "name" : "/0/model.22/Reshape_3", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_3_copy_output", "timeMs" : 2.74454, "averageMs" : 0.00633844, "medianMs" : 0.006336, "percentage" : 0.0886153 }
, { "name" : "/0/model.22/Reshape_4", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_4_copy_output", "timeMs" : 2.4913, "averageMs" : 0.00575358, "medianMs" : 0.005728, "percentage" : 0.0804386 }
, { "name" : "/0/model.22/Reshape_5", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/0/model.22/Reshape_5_copy_output", "timeMs" : 2.43866, "averageMs" : 0.005632, "medianMs" : 0.0056, "percentage" : 0.0787389 }
, { "name" : "/0/model.22/Transpose + /0/model.22/Unsqueeze_6", "timeMs" : 3.4111, "averageMs" : 0.00787784, "medianMs" : 0.007808, "percentage" : 0.110137 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to /0/model.22/Transpose + /0/model.22/Unsqueeze_6", "timeMs" : 3.51277, "averageMs" : 0.00811263, "medianMs" : 0.008064, "percentage" : 0.11342 }
, { "name" : "/0/model.22/dfl/Reshape_1", "timeMs" : 6.9063, "averageMs" : 0.0159499, "medianMs" : 0.015904, "percentage" : 0.222989 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to PWN(/0/model.22/Add)", "timeMs" : 2.89895, "averageMs" : 0.00669502, "medianMs" : 0.006688, "percentage" : 0.0936006 }
, { "name" : "PWN(/0/model.22/Add)", "timeMs" : 3.14563, "averageMs" : 0.00726474, "medianMs" : 0.007264, "percentage" : 0.101566 }
, { "name" : "PWN(/0/model.22/Add_1)", "timeMs" : 2.91149, "averageMs" : 0.006724, "medianMs" : 0.00672, "percentage" : 0.0940057 }
, { "name" : "PWN(/0/model.22/Add_2)", "timeMs" : 2.89053, "averageMs" : 0.00667559, "medianMs" : 0.00672, "percentage" : 0.0933288 }
, { "name" : "/0/model.22/Sub", "timeMs" : 4.2896, "averageMs" : 0.00990669, "medianMs" : 0.00992, "percentage" : 0.138502 }
, { "name" : "PWN(/0/model.22/Add_4)", "timeMs" : 4.2704, "averageMs" : 0.00986235, "medianMs" : 0.009952, "percentage" : 0.137882 }
, { "name" : "/0/model.22/Sub_1", "timeMs" : 4.16672, "averageMs" : 0.00962292, "medianMs" : 0.009632, "percentage" : 0.134534 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to /0/model.22/Sub_1", "timeMs" : 4.08883, "averageMs" : 0.00944304, "medianMs" : 0.009408, "percentage" : 0.13202 }
, { "name" : "/0/model.22/Transpose_1 + (Unnamed Layer* 350) [Shuffle]", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "PWN(/0/model.22/Constant_36_output_0 + (Unnamed Layer* 346) [Shuffle], PWN(/0/model.22/Add_5, /0/model.22/Div_1))", "timeMs" : 6.02016, "averageMs" : 0.0139034, "medianMs" : 0.013856, "percentage" : 0.194378 }
, { "name" : "/0/model.22/Div_1_output_0 copy", "timeMs" : 3.96192, "averageMs" : 0.00914993, "medianMs" : 0.009152, "percentage" : 0.127922 }
, { "name" : "PWN(/0/model.22/Sigmoid)", "timeMs" : 27.1587, "averageMs" : 0.0627221, "medianMs" : 0.062688, "percentage" : 0.876894 }
, { "name" : "Reformatting CopyNode for Input Tensor 1 to /0/model.22/Mul_2", "timeMs" : 3.67088, "averageMs" : 0.00847778, "medianMs" : 0.008512, "percentage" : 0.118525 }
, { "name" : "/0/model.22/Mul_2", "timeMs" : 5.63786, "averageMs" : 0.0130205, "medianMs" : 0.013024, "percentage" : 0.182034 }
, { "name" : "/0/model.22/Mul_2_output_0 copy", "timeMs" : 3.21837, "averageMs" : 0.00743273, "medianMs" : 0.007424, "percentage" : 0.103914 }
, { "name" : "Reformatting CopyNode for Input Tensor 0 to /1/Transpose", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/1/Transpose", "timeMs" : 44.2564, "averageMs" : 0.102209, "medianMs" : 0.102144, "percentage" : 1.42894 }
, { "name" : "Reformatting CopyNode for Output Tensor 0 to /1/Transpose", "timeMs" : 0, "averageMs" : 0, "medianMs" : 0, "percentage" : 0 }
, { "name" : "/1/Slice", "timeMs" : 4.83623, "averageMs" : 0.0111691, "medianMs" : 0.011136, "percentage" : 0.156151 }
, { "name" : "/1/Slice_1", "timeMs" : 27.9762, "averageMs" : 0.0646101, "medianMs" : 0.064576, "percentage" : 0.90329 }
, { "name" : "/1/ReduceMax", "timeMs" : 32.2679, "averageMs" : 0.0745217, "medianMs" : 0.074432, "percentage" : 1.04186 }
, { "name" : "/1/ArgMax", "timeMs" : 32.8336, "averageMs" : 0.0758281, "medianMs" : 0.075776, "percentage" : 1.06012 }
, { "name" : "/1/Cast", "timeMs" : 2.81644, "averageMs" : 0.00650449, "medianMs" : 0.006496, "percentage" : 0.0909368 }
]

```

Please advice

---

<div class="post-metadata">

**Author:** ![lynettez](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lynettez](https://forums.developer.nvidia.com/u/lynettez)\
**Post date:** [August 8, 2024, 6:47am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/4 "2024-08-08T06:47:52Z")

</div>

Hi, using DLA can offload some computing tasks from the GPU.  
When tasks run concurrently on both DLA and GPU, throughput can be improved.  
Your test results are as expected; the latency of running the same model on DLA is higher than on the GPU because the compute capability of DLA is lower than GPU.

---

<div class="post-metadata">

**Author:** ![rishikesan](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rishikesan](https://forums.developer.nvidia.com/u/rishikesan)\
**Post date:** [August 8, 2024, 7:32am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/5 "2024-08-08T07:32:30Z")

</div>

Thanks for the response

But as stated in the NVIDIA’s documentation , in the example it shows the throughput increases as we push more layers into the DLA

But in my case the throughput seems low when using DLA ,can you advice on this

---

<div class="post-metadata">

**Author:** ![mchi](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mchi/32/14043_2.png) [@mchi](https://forums.developer.nvidia.com/u/mchi)\
**Post date:** [August 12, 2024, 2:42am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/7 "2024-08-12T02:42:52Z")

</div>

> [@rishikesan](#):
>
> But as stated in the NVIDIA’s documentation

can you show us the doc?

---

<div class="post-metadata">

**Author:** ![yingliu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/yingliu/32/136703_2.png) [@yingliu](https://forums.developer.nvidia.com/u/yingliu)\
**Post date:** [September 3, 2024, 2:09am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/8 "2024-09-03T02:09:05Z")

</div>

**There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks**

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [September 17, 2024, 2:10am UTC](https://forums.developer.nvidia.com/t/testing-dla-using-object-detection-model-and-deepstream/302667/9 "2024-09-17T02:10:00Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.