Ubuntu 22.04
Isaac ROS 3.2
Isaac Sim 4.5
I have already completed the following four examples, and all of them run without any issues:
- Tutorial for cuMotion MoveIt Plugin with Isaac Sim
- Tutorial for RT-DETR with Isaac Sim
- Tutorial for FoundationPose with Isaac Sim
- Tutorial for ESS with Isaac Sim
Now, I would like to rebuild the Isaac Sim project used in the Tutorial for FoundationPose with Isaac Sim in order to practice adding visual components.
I am using Franka MoveIt as the base, and I followed the Sample Scene example to add visual sensors and the corresponding ActionGraph into the Franka MoveIt setup. However, every time I press Play in Isaac Sim and then run the command:
ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose_isaac_sim.launch.py
the RViz display shows the Detection3DArray in red, indicating that there is a TF issue, and the Camera is shown in orange, indicating that some required information is missing, as shown in the screenshot below.
I checked my topic configuration using the following command:
ros2 topic info /front_stereo_camera/left/camera_info --verbose
and the topic configuration appears to be correct, as shown below.
nexcobot@nexcobot:~$ ros2 topic info /front_stereo_camera/left/camera_info --verbose
Type: sensor_msgs/msg/CameraInfoPublisher count: 1
Node name: _Render_PostProcess_SDGPipeline_Replicator_NodeWriterWriter
Node namespace: /front_stereo_camera
Topic type: sensor_msgs/msg/CameraInfo
Endpoint type: PUBLISHER
GID: 01.0f.3e.af.06.37.9e.81.02.00.1a.00.00.00.05.03.00.00.00.00.00.00.00.00
QoS profile:
Reliability: RELIABLE
History (Depth): UNKNOWN
Durability: VOLATILE
Lifespan: Infinite
Deadline: Infinite
Liveliness: AUTOMATIC
Liveliness lease duration: InfiniteSubscription count: 3
Node name: resize_left_rt_detr_node
Node namespace: /
Topic type: sensor_msgs/msg/CameraInfo
Endpoint type: SUBSCRIPTION
GID: 01.0f.3e.af.55.1e.a5.12.00.00.00.00.00.00.1d.04.00.00.00.00.00.00.00.00
QoS profile:
Reliability: RELIABLE
History (Depth): UNKNOWN
Durability: VOLATILE
Lifespan: Infinite
Deadline: Infinite
Liveliness: AUTOMATIC
Liveliness lease duration: InfiniteNode name: resize_left_ess_size
Node namespace: /
Topic type: sensor_msgs/msg/CameraInfo
Endpoint type: SUBSCRIPTION
GID: 01.0f.3e.af.55.1e.a5.12.00.00.00.00.00.01.06.04.00.00.00.00.00.00.00.00
QoS profile:
Reliability: RELIABLE
History (Depth): UNKNOWN
Durability: VOLATILE
Lifespan: Infinite
Deadline: Infinite
Liveliness: AUTOMATIC
Liveliness lease duration: InfiniteNode name: rviz2
Node namespace: /
Topic type: sensor_msgs/msg/CameraInfo
Endpoint type: SUBSCRIPTION
GID: 01.0f.3e.af.57.1e.c4.58.00.00.00.00.00.00.24.04.00.00.00.00.00.00.00.00
QoS profile:
Reliability: BEST_EFFORT
History (Depth): UNKNOWN
Durability: VOLATILE
Lifespan: Infinite
Deadline: Infinite
Liveliness: AUTOMATIC
Liveliness lease duration: Infinite
After referring to the Sample Scene example, I can no longer think of how to fix this issue. I hope that someone with experience can help me. Below, I have attached the log output from running:
ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose_isaac_sim.launch.py
1768481513.4309118 [INFO] [launch]: All log files can be found below /home/admin/.ros/log/2026-01-15-20-51-53-430188-nexcobot-7225
1768481513.4310122 [INFO] [launch]: Default logging verbosity is set to INFO
1768481513.5350661 [INFO] [component_container_mt-1]: process started with pid [7237]
1768481513.5352237 [INFO] [rviz2-2]: process started with pid [7239]
1768481513.6787658 [rviz2-2] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to ‘/tmp/runtime-admin’
1768481513.9574394 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/resize_left_rt_detr_node’ in container ‘/foundationpose_container’
1768481514.3338721 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/pad_node’ in container ‘/foundationpose_container’
1768481514.3599594 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/image_to_tensor_node’ in container ‘/foundationpose_container’
1768481514.3947928 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/interleaved_to_planar_node’ in container ‘/foundationpose_container’
1768481514.4122283 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/reshape_node’ in container ‘/foundationpose_container’
1768481514.4209743 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/rtdetr_preprocessor’ in container ‘/foundationpose_container’
1768481514.4700034 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/tensor_rt’ in container ‘/foundationpose_container’
1768481514.4802563 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/rtdetr_decoder’ in container ‘/foundationpose_container’
1768481514.4903967 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/detection2_d_array_filter’ in container ‘/foundationpose_container’
1768481514.5793443 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/detection2_d_to_mask’ in container ‘/foundationpose_container’
1768481514.7139990 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/resize_mask_node’ in container ‘/foundationpose_container’
1768481514.8449085 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/resize_left_ess_size’ in container ‘/foundationpose_container’
1768481515.0110030 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/resize_right_ess_size’ in container ‘/foundationpose_container’
1768481515.0240977 [rviz2-2] [INFO] [1768481515.023958570] [rviz2]: Stereo is NOT SUPPORTED
1768481515.0243456 [rviz2-2] [INFO] [1768481515.024155089] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
1768481515.1581309 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/ess_node’ in container ‘/foundationpose_container’
1768481515.1948271 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/disparity_to_depth’ in container ‘/foundationpose_container’
1768481515.2925811 [rviz2-2] [INFO] [1768481515.292178442] [rviz2]: Stereo is NOT SUPPORTED
1768481515.6548364 [rviz2-2] [INFO] [1768481515.654224944] [rviz2]: Stereo is NOT SUPPORTED
1768481515.7716777 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/foundationpose_node’ in container ‘/foundationpose_container’
1768481516.1645129 [rviz2-2] [INFO] [1768481516.164231250] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 321.583 for reason ‘discarding message because the queue is full’
1768481518.6727605 [rviz2-2] [INFO] [1768481518.672531813] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 322.850 for reason ‘discarding message because the queue is full’
1768481521.2343411 [rviz2-2] [INFO] [1768481521.234074259] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 323.983 for reason ‘discarding message because the queue is full’
1768481523.7758296 [rviz2-2] [INFO] [1768481523.775329582] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 324.817 for reason ‘discarding message because the queue is full’
1768481526.2946279 [rviz2-2] [INFO] [1768481526.294375052] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 325.600 for reason ‘discarding message because the queue is full’
1768481528.7956421 [rviz2-2] [INFO] [1768481528.795406929] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 326.317 for reason ‘discarding message because the queue is full’
1768481531.2980602 [rviz2-2] [INFO] [1768481531.297798513] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 327.300 for reason ‘discarding message because the queue is full’
1768481533.8052533 [rviz2-2] [INFO] [1768481533.805029197] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 328.533 for reason ‘discarding message because the queue is full’
1768481536.3542154 [rviz2-2] [INFO] [1768481536.353918098] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 329.533 for reason ‘discarding message because the queue is full’
1768481538.9051032 [rviz2-2] [INFO] [1768481538.904854066] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 330.400 for reason ‘discarding message because the queue is full’
1768481541.4139938 [rviz2-2] [INFO] [1768481541.413744293] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 331.167 for reason ‘discarding message because the queue is full’
1768481543.9699771 [rviz2-2] [INFO] [1768481543.969738234] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 331.967 for reason ‘discarding message because the queue is full’
1768481546.5011284 [rviz2-2] [INFO] [1768481546.500859511] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 332.717 for reason ‘discarding message because the queue is full’
1768481549.0374916 [rviz2-2] [INFO] [1768481549.037229667] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 333.517 for reason ‘discarding message because the queue is full’
1768481551.6848564 [rviz2-2] [INFO] [1768481551.684469758] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 334.250 for reason ‘discarding message because the queue is full’
1768481554.2407849 [rviz2-2] [INFO] [1768481554.240444091] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 335.033 for reason ‘discarding message because the queue is full’
1768481556.7713282 [rviz2-2] [INFO] [1768481556.771010272] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 335.733 for reason ‘discarding message because the queue is full’
1768481559.2840796 [rviz2-2] [INFO] [1768481559.283852845] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 336.500 for reason ‘discarding message because the queue is full’
1768481561.8098528 [rviz2-2] [INFO] [1768481561.809527222] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 337.317 for reason ‘discarding message because the queue is full’
1768481564.3261821 [rviz2-2] [INFO] [1768481564.325914577] [rviz2]: Message Filter dropping message: frame ‘front_stereo_camera_left_optical’ at time 337.983 for reason ‘discarding message because the queue is full’
1768481564.6938744 [WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
1768481564.7914588 [rviz2-2] [INFO] [1768481564.693741621] [rclcpp]: signal_handler(SIGINT/SIGTERM)
1768481565.2585385 [INFO] [component_container_mt-1]: process has finished cleanly [pid 7237]
1768481565.4665599 [ERROR] [rviz2-2]: process has died [pid 7239, exit code -11, cmd ‘/opt/ros/humble/lib/rviz2/rviz2 -d /workspaces/isaac_ros-dev/install/isaac_ros_foundationpose/share/isaac_ros_foundationpose/rviz/foundationpose_isaac_sim.rviz --ros-args -r __node:=rviz2’].
This link contains my USD file.
This link provides a video of the steps I took when reproducing the issue.
