Isaac Ros Image Segmentation input data doubts

Autonomous Machines Robotics - Isaac Isaac ROS
1

Hi everyone,
I am currently working with Segment Anything using the Mobile SAM model of Issac ROS.

I have few queries like

  1. There is file name isaac_ros_assets/isaac_ros_segment_anything/quickstart_interface_specs.json in this all the keys like image , camera_info and prompt are inputs to the segmentation? What is prompt input actually please tell me more on the prompt input which the /detectnet/detections is passing is it directly data from the zed camera or any node is getting some other data then with that the /detectnet/detections topic is created?

{
“input_image”: {
“width”: 1200,
“height”: 632
},
“subscribed_topics”: {
“image”: “/image”,
“camera_info”: “/camera_info”,
“prompt”: “/detectnet/detections”
}
}

  1. If /detectnet/detections gives the detections like person or other objects directly the segment everything will not work right ? only on the detections that is the class id which the topic data is giving will be segmented right?

  2. I want to know the exact creation of the /detectnet/detections topic data

ros2 bag info segment_anything_sample_data/

Files: segment_anything_sample_data_0.db3
Bag size: 65.3 MiB
Storage id: sqlite3
Duration: 2.891s
Start: Feb 16 2024 17:54:25.952 (1708086265.952)
End: Feb 16 2024 17:54:28.843 (1708086268.843)
Messages: 88
Topic information: Topic: /camera_info | Type: sensor_msgs/msg/CameraInfo | Count: 29 | Serialization Format: cdr
Topic: /detectnet/detections | Type: vision_msgs/msg/Detection2DArray | Count: 29 | Serialization Format: cdr
Topic: /image | Type: sensor_msgs/msg/Image | Count: 30 | Serialization Format: cdr

Please help

2

Hi @anishm1 ,

Let me reply question by question.

  1. The prompt is topic published by object detection decode node. In this rosbag sample, we will use detectnet object detection to determine the image space bounding box of an object of interest to use as the prompt for Segment Anything to create an image segmentation mask for.

  2. It’s image space bounding box of an object. The type depends on the model used in object detection.

  3. About the topic /detectnet/detections, please refer to “ROS 2 Graph Configuration” and “ROS Topics Published” in isaac_ros_detectnet — isaac_ros_docs documentation

I hope my message is helpful. If not, please let me know.