Questions regarding foundation pose tracking results

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hi. I think Foundation pose is so cool. Thanks for making it.

Hello,

I am currently using the pre-trained banana class from RT-DETR for foundation pose tracking. However, I am noticing some inaccuracies, which is why I am reaching out for assistance.

In the attached video, particularly towards the end, you can see the 3D detection result moving independently, which seems off. What could be causing this issue?

Additionally, I am interested in training RT-DETR on custom data for use in foundation pose tracking. Is there any manual or reference material available for this process?

2

Hi @hirship1234

Thank you for your video. I forward this issue to the engineering.
Meanwhile, can you share some more detailed information?

  • hardware setup
  • which demo are you running?
  • changing background: Is the issue the same?

Thank you in advance,
Raffaello

3

Thank you for @Raffaello reply.

  1. Hardware Setup

I am using the Realsense D455 camera along with a desktop equipped with an RTX 4070 Ti Super 16GB GPU. For the Intel RealSense setup, I followed the method provided in the Isaac ROS documentation.

  1. Demo

For the demo, I created the 3D model as instructed by the Isaac ROS documentation and used the Foundation Pose Tracking demo.

  1. Background

I haven’t changed the background yet, but since the application will be for a real robot task, I am considering the presence of surrounding objects, as it may resemble the current video setup.