I’m working on a project streaming two usb camera streams from a Xavier agx to another laptop for viewing. The agx is performing object detection using a custom model then publishing the streams with labeled boxes and data about the object detected over ros. I considered webrtc for this but settled on ROS as it’s easier for to stand up given I have past ROS experience.
The challenge I’m running into is using Deepstream in 20.04 with ros2. The GitHub for Ros2 Deepstream bodes shows compatibility with 18.04 only and hasn’t been updated in three years. Before I dive headlong into making this work, is there another way I’m supposed to do this? I’m new to Deepstream and still reconciling where Isaac ands and it begins for some of my work (not a criticism, I’m just still figuring this out).
Hi @dbrownrxvs0, what kind of model are you using for object detection? I would start with what inferencing library your model is compatible with (whether it be PyTorch, DeepStream, isaac_ros_object_detection, ros_deep_learning, YOLO, ect) and go from there.
For new ROS2 applications on Jetson, I would consider using ISAAC ROS going forward. Given your experience with ROS however, you can also just roll your own ROS node around the model inferencing code that you already have.
@dusty_nv Thanks for the advice. Nice to speak with you too, I feel like I’ve read a few hundred pages of your work by now :)
This is a greenfield project. I was looking at yolov8. I haven’t done an object detection project in a couple years so I’m having to get up to speed on the changes in the space. This is just two cameras on an AGX Xavier communicating to other nodes over LAN and doesn’t have any serious real-time constraints. After this is another project (also greenfield) requiring real-time (sub 100ms start to finish) streaming over the internet. My initial plan was to use the same approach for both, but the first one is running up against a deadline. I’m going to read more into Deepstream for project #2. Any pointers are highly welcome, particularly regarding latency reduction in the detection and transfer side. I also know ROS wasn’t always the best for moving streaming video, so Isaac is really interesting. I’ll keep reading.