Isaac ros yolov8 zedxm live feed

Hello, I am working on a project that requires me to use a custom trained object detection algorithm and have it publish the bounding box coordinates along with the class on to a ros topic so i can control a vehicle based on that information. I have been following the instructions in the isaac_ros_yolov8 tutorial and i was wondering if anyone could guide me on how i can inject a live feed instead of a single image like in the tutorial.

thank you.

Hi @castej10

You can use the demo linked above to test using other inputs. The demo runs all nodes and accepts such an input on the /image topic.

cd /workspaces/isaac_ros-dev && \
   ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/tmp/yolov8s.onnx engine_file_path:=/tmp/yolov8s.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45

This is a visualization of the ROS2 graph for this demo.

If you want to use a ZEDXMini, you can remap a topic from the camera and rename it to /image

1 Like

Thank you that helped me out immensely. i was able to publish to the /image topic with a custom node but the issue im running into now is that the topic that is able to provide bgr8 (which is what /image is expecting) /zed/zed_node/left_raw/image_raw_color/compressed is publishing messages of type sensor_msgs/msg/CompressedImage , not sensor_msgs/msg/Image. This discrepancy is causing an issue with the subscription and publishing, as isaac_ros_yolov8 node is set up to handle Image messages, not CompressedImage messages. if i try then to decompress the image with my custom bridge nose shown here:

import rclpy
from rclpy.node import Node
from sensor_msgs.msg import CompressedImage, Image
from cv_bridge import CvBridge

class ImageDecompressor(Node):
    def __init__(self):
        super().__init__('image_decompressor')
        self.bridge = CvBridge()
        self.subscription = self.create_subscription(
            CompressedImage,
            '/zed/zed_node/left_raw/image_raw_color/compressed',
            self.callback,
            10)
        self.publisher = self.create_publisher(Image, '/image', 10)

    def callback(self, msg):
        # Convert compressed image to OpenCV format
        cv_image = self.bridge.compressed_imgmsg_to_cv2(msg, desired_encoding='bgr8')

        # Convert OpenCV image back to ROS Image message and publish
        image_message = self.bridge.cv2_to_imgmsg(cv_image, encoding='bgr8')
        image_message.header = msg.header  # Copy header to maintain timestamp and frame info
        self.publisher.publish(image_message)

def main(args=None):
    rclpy.init(args=args)
    node = ImageDecompressor()
    rclpy.spin(node)
    node.destroy_node()
    rclpy.shutdown()

if __name__ == '__main__':
    main()

it actually works but i get a weird output that seems to have the bounding boxes shifted over to the left, and the further you are away from the camera the more to the left the bounding boxes get shifted. here is a picture show what im talking about as its hard to describe.


as you can see here the bounding boxes for the people in the back are pushed so far to the left that they are covering the person taking the picture, and the bounding box for the person taking the picture is to his left by his hands and so on. the further back you are the more to the left the bounding boxes appear. the only way i can test to see if its because of the decompression being done by cv2 is to feed the compressed files but i cannot.

is there a way of configuring the isaac_ros_yolov8 node to subscribe to a different, unique topic for its input images, rather than /image . or can we change the type that its expecting to CompressedImage?

thank you

Happy to see everthing works :-)

yes, sure!

I suggest rewriting your launch file and remapping the image subscriber with a new name. I also recommend using our package Isaac ROS Image Pipeline — isaac_ros_docs documentation, where you can find different packages for image processing.

1 Like

thank you @Raffaello! it looks like im going to have to use the Isaac ROS Image Proc since simply remapping the nodes wont change the type they are expecting. but i was looking at it and the ZedX mini camera isnt in the other supported cameras list and realsense is the only other camera mentioned. Does this mean ZedX mini isnt supported or simply not mentioned.

thanks again for your help, you have helped me understand so much.

Regarding the camera, we have created the SQA. You can check this page for more information: Sensors Setup — isaac_ros_docs documentation

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.