I have been following the quickstart for the Foundation Pose implementation which is part of the Isaac ROS Pose Estimation package. While I was able to replicate the detection of the object in the Rosbag, I was not able to perform the same for using the Isaac ROS RT-DETR object detection. With an error on the RT-DETR visualization script:
The foundation pose quickstart using an Realsense camera implementation did not provide any results . I am able to visualize the images acquired by the camera with realsense-viewer but as far, as using the Foundation Pose model it gave no output, and no clear error message in the logs.
One other more general question, would be on the necessary setup to identify an novel object with this package, while Isaac ROS FoundationPose is capable of dealing with novel object, the Isaac ROS RT-DETR which is responsible for providing the segmentation mask requires to be trained for the objects I want to detect? Did I understood this correctly?
Hello there,
Regarding the initial error, I just downgraded the numpy version. I am running in Ubuntu 20.04 with a RTX 3090, I am still having trouble understanding how to detect custom objects with the FoundationPose. I do not want to use DOPE, I want to use the suggested pipeline of the quickstart of FoundationPose, but from what I understood one of the modules of this pipelines consists on using ROS RT-DETR to attain the 2D bounding box detection, which will then be fed to the Selector node.
My question is how can I change the pipeline (launch file) to perform the detection of custom objects that are not included in the SyntheticaDETR dataset. For example I already have yolov8 weights for custom detection, how can I change the FoundationPose pipeline to detect these objects? Besides feeding the necessary inputs: image, depth image, camera info and obj model, of course.
