Isaac_ros_foundationpose Quickstart Issues with RealSense Cameras

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hello Nvidia Developer Forum,

I am having trouble with successfully completing the isaac_ros_foundationpose Quickstart as the realsense cameras keep failing to launch.

I have been successful in completing the:

  • Isaac ROS RealSense Setup and the realsense-viewer tool works correctly inside the container
  • isaac_ros_apriltag quickstart for both Rosbag and realsense
  • isaac_ros_rtdetr quickstart for both Rosbag and realsense
  • isaac_ros_foundationpose for only the Rosbag.

During these I have had some issues with the realsense cameras but after repeatedly following the guides I have managed to get them working.

However, now running through the isaac_ros_foundationpose quickstart many times with the realsense launch file and looking through many support posts I haven’t been able to successfully complete this quick start.

Following the tutorials and running the launch file produces the following output:
foundationPose Launch Output.txt (54.0 KB)

I am running these on a desktop with an RTX4090. I have access to multiple realsense cameras of class D435 and D455.

Could I ask for support to solve this issue?

2

Hi @benjamin.gifford

Thank you for your post.

From your attached, there are two warnings with hardware error from realsense.

[realsense2_camera]: XXX Hardware Notification:Right MIPI error,1.74601e+12,Error,Hardware Error
[realsense2_camera]: Hardware Notification:Right MIPI error,1.74601e+12,Error,Hardware Error

I don’t have detail about these error. To better assist you, please help check…

  • Is this hardware error a consistent message every time the demo is failed to launch?
  • Did you observe the same with other realsense camera?
  • If multiple realsense cameras were connected, can you try connecting just one camera module?
  • Have you tried changing USB port, USB cable and removing USB Hub if any?

Best,
Ahung

3

Hi Ahung,

Thank you for your reply. I hope I have collected all the information you might need to support below.

foundationPose Launch Output (1) 14:27_08_05_25.txt (54.5 KB)

Output (1) shows the output after following the quickstart guide and running the foundation pose launch. I cannot see any output matching what you highlighted above.

foundationPose Launch Output (2) 14:29_08_05_25.txt (51.0 KB)

Output (2) shows the output after rerunning the launch file after stopping the previous.

Screenshot from 2025-05-08 14-36-11
Screenshot from 2025-05-08 14-36-111920×1080 178 KB

foundationPose Launch Output (3) 14:40_08_05_25.txt (55.1 KB)
Screenshot from 2025-05-08 14-41-38
Screenshot from 2025-05-08 14-41-381920×1080 439 KB

This first screen shot shows the realsense camera working using the realsense-viewer after rebuilding the container. I then ran the launch file, shown in output (3). Then the second screen shot shows the realsense-viewer no long being able to detect the realsense camera.

foundationPose Launch Output (4) 14:56_08_05_25.txt (55.0 KB)

Finally, I have changed the camera, the cable and the USB port on the PC and captured the Output (4). The output here is almost identical to Output (1) apart from the obvious changes expected with changing to the D455 with the IMU.

So to answer your questions.

  1. yes it seems the output is consistent. Output (1) and (4) are almost identical. (3) has similar output but in a slightly different order. Output (2) is different as it seems the USB isn’t being closed properly when the launch file crashes, see lines 14-22.

  2. Yes as shown above.

  3. All outputs show with only 1 realsense connected

  4. USB port and USB cable changed for output 4. No USB hub in use.

One last thing to draw your attention to is line 307 in output (1). The realsense node outputs a warning with no informaiton as shown below:
[component_container_mt-1] [INFO] [1746710559.150581384] [realsense2_camera]: RealSense Node Is Up!
[component_container_mt-1] [WARN] [1746710559.305934948] [realsense2_camera]:

I hope this information helps and I look forward to hearing your response.

Many Thanks,
Ben

4

Hi @benjamin.gifford

Thank you for the details.

Your new attached logs doesn’t have the hardware error. I think that demo was working.
Are you able to visualize the result?

1. Open a new terminal inside the Docker container:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh

2. launch RViz2 to visualize the output

rviz2 -d  $(ros2 pkg prefix isaac_ros_foundationpose --share)/rviz/foundationpose_realsense.rviz

3. You should see a visualization of the 3D pose estimate.

Best,
Ahung

5

@benjamin.gifford Empirically, Ive observed this error to consistently get solved if I just unplug and plug the camera.

It happens on other devices too, its not a jetson specific issue, but it does seem to occur more often with a jetson.