Realsense, wsl2, and foundation pose

Congrats on Isaac ROS 3.0. Very nice work. Things work very smoothly on the ORIN AGX including the FoundationPose and FoundationPose tracking reference flows.

As an experiment I was interested in the performance on x86 running wsl2.0 My system is i7, 13th generation, 64GB, 4080. The complicating factor is wsl2, and I am running into issues.

Specifically, I am running into an issue with the realsense when running the FoundationPose and FoundationPose tracking reference workflows. In the case of the tracking reference flow, I am getting repeated [WARN] [1718227867.271321262] [realsense2_camera]: XXX Hardware Notification:Frames didn’t arrived within 5 seconds,1.71823e+12,Warn,Frames Timeout, and of course, no detections. With the pure FoundationPose reference flow, I get detections, but the peformance is considerably worse than the ORIN.

The RtDetr reference flow performance is good, so maybe this is associated with getting the depth map from the camera. The depth performance seems good in the viewer.

Any ideas? I do get that I am making life difficult for myself by using wsl2.

Hmmm… I am seeing the same on the viewer when accessing depth, so that must be the issue and not a Isaac ROS issue (although the viewer is running in the ros docker container.

Hi! Since things worked on the Jetson AGX Orin, one idea is to record a rosbag with the required input topics and test that on x86. This would help check if it’s a hardware issue.

@asawareeb. For foundationpose, I am recording the data with realsense 435i and I have same topic name and message type. I also got same mustard bottle. If I run quickstart rosbag, it is working but when record my own rosbag, it is not showing anything in rviz. I am able to run node successfully but no output in rviz. The rqt_graph is same from both rosbags. Is there any procedure I need to follow while recording the data? I am attaching the screenshots.
I have RTX 6000 and I was able to run foundationpose from git on my custom object but not able to run Issac_ros_foundationpose. If you need any more details. please feel free to ask.
foundationpose_debug.zip (2.8 MB)

Could you please check if the camera intrinsics match those from the quickstart rosbag? You can do this by running ros2 topic echo /camera_info_rect while playing your rosbag and comparing it with the quickstart rosbag.

@asawareeb. My intrinsic was not matching the values of given rosbag intrinsic. I am using realsense 435i. I am working with two models foundationpose and rtdetr. Both has different intrinsic values in their quickstart bags. I think rtdetr rosbag is recorded with realsense and foundationpose rosbag is recorded with kinect. I have recorded rosbag with both camera info as quickstart bag and tried running both rosbag on foundationpose and rtdetr but it is still not working. I am using same mustard sauce and mac_cheese box. I am attaching few rosbags which are recorded with same intrinsic and also attaching script I am using to publish the data.
If you have any standard process for recording the data, can you share that or make tutorial post for that? I think I might be recording data wrong way or missing something. I have similar topic names and message type.
If you need any more data, feel free to ask.
https://drive.google.com/drive/folders/19ogV6ueRhTLTCZFAqxThvlWK12AnV-l5?usp=sharing