# Improved DeepStream for YOLO models **URL:** \ **Category:** DeepStream SDK\ **Created:** [February 14, 2022, 1:26am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368 "2022-02-14T01:26:50Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [February 14, 2022, 1:26am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/1 "2022-02-14T01:26:50Z")
Hi, I would like to share my work improving the DeepStream for YOLO models provided by NVIDIA. Link: [GitHub - marcoslucianops/DeepStream-Yolo: NVIDIA DeepStream SDK 8.0 / 7.1 / 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models](https://github.com/marcoslucianops/DeepStream-Yolo) ### Improvements on this repository - Darknet CFG params parser (no need to edit nvdsparsebbox\_Yolo.cpp or another file) - Support for new\_coords, beta\_nms and scale\_x\_y params - Support for new models - Support for new layers - Support for new activations - Support for convolutional groups - Support for INT8 calibration - Support for non square models - Support for implicit and channel layers (YOLOR) - YOLOv5 6.0 native support - Initial YOLOR native support - Models benchmarks ### Future updates - New documentation for multiple models - DeepStream tutorials - Native PP-YOLO support - GPU NMS - Dynamic batch-size ### Tested models - [Darknet YOLO](https://github.com/AlexeyAB/darknet) - [YOLOv5 6.0](https://github.com/ultralytics/yolov5) - [YOLOR](https://github.com/WongKinYiu/yolor) - [MobileNet-YOLO](https://github.com/dog-qiuqiu/MobileNet-Yolo) - [YOLO-Fastest](https://github.com/dog-qiuqiu/Yolo-Fastest) ### Benchmarks Board: NVIDIA GTX 1050 4GB (Mobile) ##### YOLOR-CSP performance comparison | | DeepStream | PyTorch | | --- | --- | --- | | FPS (without display) | 13.32 | 10.07 | | FPS (with display) | 12.63 | 9.41 | ##### YOLOv5n performance comparison | | DeepStream | TensorRTx | Ultralytics | | --- | --- | --- | --- | | FPS (without display) | 110.25 | 87.42 | 97.19 | | FPS (with display) | 105.62 | 73.07 | 50.37 | > **More** > > | DeepStream | Precision | Resolution | IoU=0.5:0.95 | IoU=0.5 | IoU=0.75 | FPS > (without display) | > | --- | --- | --- | --- | --- | --- | --- | > | YOLOR-P6 | FP32 | 1280 | 0.478 | 0.663 | 0.519 | 5.53 | > | YOLOR-CSP-X\* | FP32 | 640 | 0.473 | 0.664 | 0.513 | 7.59 | > | YOLOR-CSP-X | FP32 | 640 | 0.470 | 0.661 | 0.507 | 7.52 | > | YOLOR-CSP\* | FP32 | 640 | 0.459 | 0.652 | 0.496 | 13.28 | > | YOLOR-CSP | FP32 | 640 | 0.449 | 0.639 | 0.483 | 13.32 | > | YOLOv5x6 6.0 | FP32 | 1280 | 0.504 | 0.681 | 0.547 | 2.22 | > | YOLOv5l6 6.0 | FP32 | 1280 | 0.492 | 0.670 | 0.535 | 4.05 | > | YOLOv5m6 6.0 | FP32 | 1280 | 0.463 | 0.642 | 0.504 | 7.54 | > | YOLOv5s6 6.0 | FP32 | 1280 | 0.394 | 0.572 | 0.424 | 18.64 | > | YOLOv5n6 6.0 | FP32 | 1280 | 0.294 | 0.452 | 0.314 | 26.94 | > | YOLOv5x 6.0 | FP32 | 640 | 0.469 | 0.654 | 0.509 | 8.24 | > | YOLOv5l 6.0 | FP32 | 640 | 0.450 | 0.634 | 0.487 | 14.96 | > | YOLOv5m 6.0 | FP32 | 640 | 0.415 | 0.601 | 0.448 | 28.30 | > | YOLOv5s 6.0 | FP32 | 640 | 0.334 | 0.516 | 0.355 | 63.55 | > | YOLOv5n 6.0 | FP32 | 640 | 0.250 | 0.417 | 0.260 | 110.25 | > | YOLOv4-P6 | FP32 | 1280 | 0.499 | 0.685 | 0.542 | 2.57 | > | YOLOv4-P5 | FP32 | 896 | 0.472 | 0.659 | 0.513 | 5.48 | > | YOLOv4-CSP-X-SWISH | FP32 | 640 | 0.473 | 0.664 | 0.513 | 7.51 | > | YOLOv4-CSP-SWISH | FP32 | 640 | 0.459 | 0.652 | 0.496 | 13.13 | > | YOLOv4x-MISH | FP32 | 640 | 0.459 | 0.650 | 0.495 | 7.53 | > | YOLOv4-CSP | FP32 | 640 | 0.440 | 0.632 | 0.474 | 13.19 | > | YOLOv4 | FP32 | 608 | 0.498 | 0.740 | 0.549 | 12.18 | > | YOLOv4-Tiny | FP32 | 416 | 0.215 | 0.403 | 0.206 | 201.20 | > | YOLOv3-SPP | FP32 | 608 | 0.411 | 0.686 | 0.433 | 12.22 | > | YOLOv3-Tiny-PRN | FP32 | 416 | 0.167 | 0.382 | 0.125 | 277.14 | > | YOLOv3 | FP32 | 608 | 0.377 | 0.672 | 0.385 | 12.51 | > | YOLOv3-Tiny | FP32 | 416 | 0.095 | 0.203 | 0.079 | 218.42 | > | YOLOv2 | FP32 | 608 | 0.286 | 0.541 | 0.273 | 25.28 | > | YOLOv2-Tiny | FP32 | 416 | 0.102 | 0.258 | 0.061 | 231.36 | ---
**Author:** ![mchi](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mchi/32/14043_2.png) [@mchi](https://forums.developer.nvidia.com/u/mchi)\ **Post date:** [February 14, 2022, 9:51am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/3 "2022-02-14T09:51:01Z")
Amazing! Really appreciate for your work on DeepStream! This is helpful, I’ll share this internally! One question, in your test table, YoloV4/3/2 are from [Darknet YOLO](https://github.com/AlexeyAB/darknet) , right? ---
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [February 14, 2022, 9:59am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/4 "2022-02-14T09:59:03Z")
> [@mchi](#): > > One question, in your test table, YoloV4/3/2 are from [Darknet YOLO](https://github.com/AlexeyAB/darknet) , right? Yes ---
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [February 17, 2022, 6:34pm UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/5 "2022-02-17T18:34:19Z")
**Moved from CPU to GPU to get better performance** ### Results 4x faster inference in AGX using YOLOv5n model in FP16 mode | CPU YOLO Decoder | GPU YOLO Decoder | | --- | --- | | ![image](https://global.discourse-cdn.com/nvidia/original/3X/1/f/1f00d577b9ee72eeffbd1a75d68aaf56b9087eef.png) | ![image](https://global.discourse-cdn.com/nvidia/original/3X/5/5/556052746c5bc48a999ed426fed1e9db3eecc862.png) | ---
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [February 22, 2022, 2:54am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/6 "2022-02-22T02:54:38Z")
Update: - GPU YOLO Decoder (moved from CPU to GPU to get better performance) [#138](https://github.com/marcoslucianops/DeepStream-Yolo/issues/138) - Improved NMS [#142](https://github.com/marcoslucianops/DeepStream-Yolo/issues/142) ---
**Author:** ![music1913](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@music1913](https://forums.developer.nvidia.com/u/music1913)\ **Post date:** [March 10, 2022, 6:18am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/8 "2022-03-10T06:18:47Z")
I’m using `TAO` to trained a `yolo_v4_tiny` model and exported to `.etlt` file, I didn’t see too much info for how to deployed it to Jetson Nano (also the DS6 samples does not include samples for YOLO), does the deepstream6 not support or? ---
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [March 10, 2022, 3:21pm UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/9 "2022-03-10T15:21:57Z")
For YOLOv4 trained in TAO you should you [deepsteam\_tao\_apps](https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps) ---
**Author:** ![music1913](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@music1913](https://forums.developer.nvidia.com/u/music1913)\ **Post date:** [March 11, 2022, 12:34am UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/10 "2022-03-11T00:34:23Z")
@marcoslucianops thanks. Your work: [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) is a improvement to [deepstream\_tao\_apps](https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps), is that correct? ---
**Author:** ![marcoslucianops](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/marcoslucianops/32/15885_2.png) [@marcoslucianops](https://forums.developer.nvidia.com/u/marcoslucianops)\ **Post date:** [March 11, 2022, 12:00pm UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/11 "2022-03-11T12:00:51Z")
No, it’s a improvement of **objectDetector\_Yolo**. ---
**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\ **Post date:** [March 25, 2022, 12:01pm UTC](https://forums.developer.nvidia.com/t/improved-deepstream-for-yolo-models/203368/12 "2022-03-25T12:01:09Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.