Modifying YOLOv8 Decoder Node for Segmentation Mask Extraction

Robotics & Edge Computing Robotics - Isaac Isaac ROS
, ,
1

I am currently exploring the use of a segmentation model for my project. To integrate this model, it appears necessary for me to adapt or develop a new decoder node. The current decoder node in the isaac_ros_yolov8 repository is designed for extracting bounding boxes, which does not align with my requirement for segmentation mask extraction.

Would it be feasible to modify the YOLOv8 decoder node, specifically the section responsible for bounding box extraction, to enable segmentation mask extraction instead? Or does this task require a more comprehensive overhaul than initially anticipated?

Any guidance or recommendations on how to approach this modification would be greatly appreciated.

I’ve also asked this on the github repo: Modifying YOLOv8 Decoder Node for Segmentation Mask Extraction · Issue #36 · NVIDIA-ISAAC-ROS/isaac_ros_object_detection · GitHub

2

Hi navier,

Isaac ROS Object Detection currently supports only the bounding box message with the vision_msgs/Detection2DArray message.

I report your request to the engineering for future development.

Best,
Raffaello

3

YOLOv8 and packages in Isaac ROS Object Detection in general are for image-space bounding box detection algorithms. Isaac ROS Image Segmentation has packages for per-pixel labels which is what I think you’re looking for using PeopleSemSegNet and more. If you have an image-space bounding box algorithm and want to get a segmentation mask from it, Segment Anything would be useful and we’re working on adding support for this model.