I’m a young developer who recently started working with an NVIDIA Jetson Orin AGX development kit. I’m relatively new to this area, but I’m excited to explore the possibilities of this powerful hardware for a project I’m working on.
My goal is to run YOLO object detection on 8 video streams simultaneously. The video streams will be coming either through RTSP feeds or USB webcams. I’ve already trained a custom model using YOLOv5s, and I’m confident that doing inference at 640x640 resolution will work well for my application.
However, I’m not sure what kind of performance I can expect, particularly in terms of frame rate. My target is to achieve at least 20 FPS across all streams. I’m also open to using fewer cameras or deploying additional Orin devices if needed to meet this requirement.
Here are a few specific questions I have:
Feasibility: Is it possible to run YOLO (and if so, which version) on 8 video streams simultaneously on the Jetson Orin AGX? What kind of performance can I realistically expect?
Best YOLO Version: Which version of YOLO would be the most suitable for this task? Should I stick with YOLOv5s, or is there a better option considering the hardware?
Setup: What’s the best approach to set up and manage multiple video streams for YOLO inference? Would using Docker containers be the right choice, or is there a better method?
Optimization: Are there any tips, tools, or techniques for optimizing YOLO on the Orin AGX, especially when dealing with multiple video streams?
Frame Rate Expectations: Based on your experience, what kind of FPS can I expect when running YOLO on 8 video streams at 640x640 resolution on the Orin AGX?
Scalability: If it’s not feasible to achieve the desired performance with 8 streams, what would be the best approach—reducing the number of cameras or adding more Orin devices?
I’m really eager to learn and would greatly appreciate any advice, resources, or examples you can share to help me get started on this. Thanks in advance for your help!
1.
Yes, it’s recommended to use our Deepstream SDK for multiple-stream inference.
We don’t have exact YOLOv5s performance data but the below sample has the YOLOv7 with 16 input streams for your reference:
2.
This is more related to the task/accuracy you used.
Most YOLO series models can run fast on Orin.
3.
It’s recommended to use JetPack 6.0 with Deepstream 7.0.
Deepstream can be set up locally or with docker depends your preference. Both should work well.
Here is the Deepstream container link for your reference:
4. The most complicated task in your use case is resource control.
For example, how to integrate 8 streams into a batch and feed into GPU together to leverage the resource.
This is already well-optimized in Deepstream.
5. We now don’t have the YOLO model in our benchmark table.
But below are some LLM-related model fps for your reference:
6. Deesptream can add/remove streams via script.
If cloud is required in your use case, our Jetson Platform Sevevice can also help you to do so:
Thank you for the update. At the moment, I am limited to using JetPack version 5.1.3 due to dependencies on specific hardware drivers essential to my project. However, migrating to version 6.0 is certainly in the plans for the future.
Could you please clarify whether DeepStream is compatible with JetPack 5.1.3? Are there significant changes between versions 5.1.3 and 6.0 that I should be aware of?
From what I understand, I can continue developing my object detection model using YOLO, and later export it to ONNX to integrate with DeepStream. Is this a feasible approach?
I was particularly impressed by the capabilities shown in this video YouTube link, where the system handles six 1080p streams with inference done at 1080p.
In my project, I need to divide tasks among different teams—one focusing on dataset preparation and model training (we have tested YOLO so far), and the other on ensuring the models can be executed at the required speed.
It’s crucial for me to confirm that our approach with YOLO and dataset labeling is a sound investment and not a potential waste of resources.
Deepstream is also available on JetPack 5.1.3 and should have the features you need.
The simplest way is to test the 8 streams with the default model on Orin directly.
Once the performance is good, you can prepare the data and customize the model.
We are not available to share third-party links here.
But search with yolov5s deepstream keyword, you should find some examples for YOLOv5s+multi-streams.