Realsense D456 on Isaac Ros 3 not working with Visual Slam

Autonomous Machines Robotics - Isaac Isaac ROS
1

Hello,

I am using Isaac ROS 3.0 and have a few D456 Realsense cameras connected to my Jetson Orin AGX. I am encountering an issue with running Visual Slam on my Realsense cameras, and I have been using this guide to get me setup: Isaac ROS Visual Slam Quick Start

Specifically, when I run isaac_ros_examples.launch.py from the guide and open the package in rviz2, the topics are published and they are all empty. I found another post on the forum where I found that I need to specify the device type in the realsense_camera_node ( Issac ros visual slam and realsense not working out of the box - Robotics - Isaac / Isaac ROS - NVIDIA Developer Forums). But that only showed the feed of the left and right image feed of the Realsense cameras but none of the point cloud or other topics work. I included my ROS topic list below as well as the output from running the example launch file mentioned earlier. I’d really appreciate any assistance!

admin@ubuntu-rover-RK:/workspaces/isaac_ros-dev$ ros2 topic list
/clicked_point
/diagnostics
/goal_pose
/infra1/image_rect_raw_mono
/infra1/image_rect_raw_mono/nitros
/infra2/image_rect_raw_mono
/infra2/image_rect_raw_mono/nitros
/initialpose
/left/camera_info_rect
/left/image_rect
/left/image_rect/nitros
/left/image_rect_mono
/left/image_rect_mono/nitros
/parameter_events
/realsense2_camera/accel/imu_info
/realsense2_camera/accel/metadata
/realsense2_camera/accel/sample
/realsense2_camera/extrinsics/depth_to_accel
/realsense2_camera/extrinsics/depth_to_gyro
/realsense2_camera/extrinsics/depth_to_infra1
/realsense2_camera/extrinsics/depth_to_infra2
/realsense2_camera/gyro/imu_info
/realsense2_camera/gyro/metadata
/realsense2_camera/gyro/sample
/realsense2_camera/imu
/realsense2_camera/infra1/camera_info
/realsense2_camera/infra1/image_rect_raw
/realsense2_camera/infra1/image_rect_raw/compressed
/realsense2_camera/infra1/image_rect_raw/compressedDepth
/realsense2_camera/infra1/image_rect_raw/theora
/realsense2_camera/infra1/metadata
/realsense2_camera/infra2/camera_info
/realsense2_camera/infra2/image_rect_raw
/realsense2_camera/infra2/image_rect_raw/compressed
/realsense2_camera/infra2/image_rect_raw/compressedDepth
/realsense2_camera/infra2/image_rect_raw/theora
/realsense2_camera/infra2/metadata
/right/camera_info_rect
/right/image_rect
/right/image_rect/nitros
/right/image_rect_mono
/right/image_rect_mono/nitros
/rosout
/tf
/tf_static
/visual_slam/status
/visual_slam/tracking/odometry
/visual_slam/tracking/slam_path
/visual_slam/tracking/vo_path
/visual_slam/tracking/vo_pose
/visual_slam/tracking/vo_pose_covariance
/visual_slam/vis/gravity
/visual_slam/vis/landmarks_cloud
/visual_slam/vis/localizer
/visual_slam/vis/localizer_loop_closure_cloud
/visual_slam/vis/localizer_map_cloud
/visual_slam/vis/localizer_observations_cloud
/visual_slam/vis/loop_closure_cloud
/visual_slam/vis/observations_cloud
/visual_slam/vis/pose_graph_edges
/visual_slam/vis/pose_graph_edges2
/visual_slam/vis/pose_graph_nodes
/visual_slam/vis/slam_odometry
/visual_slam/vis/velocity

[component_container_mt-1] [INFO] [1721413908.283349025] [realsense2_camera]: Checking new devices…
[component_container_mt-1] [INFO] [1721413908.687958318] [realsense2_camera]: Device with serial number 309122300173 was found.
[component_container_mt-1]
[component_container_mt-1] [INFO] [1721413908.688115677] [realsense2_camera]: Device with physical ID 2-3.1-18 was found.
[component_container_mt-1] [INFO] [1721413908.688134062] [realsense2_camera]: Device with name Intel RealSense D456 was found.
[component_container_mt-1] [INFO] [1721413908.688536028] [realsense2_camera]: Device with port number 2-3.1 was found.
[component_container_mt-1] [INFO] [1721413908.688566616] [realsense2_camera]: Device USB type: 3.2
[component_container_mt-1] [INFO] [1721413908.688636439] [realsense2_camera]: getParameters…
[component_container_mt-1] [WARN] [1721413908.690262207] [realsense2_camera]: For the ‘unite_imu_method’ param update to take effect, re-enable either gyro or accel stream.
[component_container_mt-1] [INFO] [1721413908.690386832] [realsense2_camera]: JSON file is not provided
[component_container_mt-1] [INFO] [1721413908.690415434] [realsense2_camera]: Device Name: Intel RealSense D456
[component_container_mt-1] [INFO] [1721413908.690432698] [realsense2_camera]: Device Serial No: 309122300173
[component_container_mt-1] [INFO] [1721413908.690446310] [realsense2_camera]: Device physical port: 2-3.1-18
[component_container_mt-1] [INFO] [1721413908.690457232] [realsense2_camera]: Device FW version: 5.16.0.1
[component_container_mt-1] [INFO] [1721413908.690467065] [realsense2_camera]: Device Product ID: 0x0B5C
[component_container_mt-1] [INFO] [1721413908.690477603] [realsense2_camera]: Sync Mode: Off
[component_container_mt-1] 19/07 11:31:51,993 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:52,004 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:52,015 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:52,027 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] [WARN] [1721413915.122202977] [realsense2_camera]: re-enable the stream for the change to take effect.
[component_container_mt-1] [INFO] [1721413915.136903173] [realsense2_camera]: Stopping Sensor: Depth Module
[component_container_mt-1] [INFO] [1721413915.137053287] [realsense2_camera]: Stopping Sensor: Motion Module
[component_container_mt-1] [INFO] [1721413915.435568388] [realsense2_camera]: Starting Sensor: Depth Module
[component_container_mt-1] [INFO] [1721413915.540342425] [realsense2_camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 848, Height: 480, FPS: 30
[component_container_mt-1] [INFO] [1721413915.540496319] [realsense2_camera]: Open profile: stream_type: Infra(2), Format: Y8, Width: 848, Height: 480, FPS: 30
[component_container_mt-1] 19/07 11:31:55,543 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,554 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,566 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,577 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,591 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,602 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,618 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,630 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,641 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,652 WARNING [281470371031264] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1] 19/07 11:31:55,664 WARNING [281471821801696] (ds-calib-parsers.cpp:35) IMU Calibration is not available, default intrinsic and extrinsic will be used.
[component_container_mt-1] [INFO] [1721413915.672731628] [realsense2_camera]: Starting Sensor: Motion Module
[component_container_mt-1] [INFO] [1721413915.703436095] [realsense2_camera]: Open profile: stream_type: Accel(0)Format: MOTION_XYZ32F, FPS: 200
[component_container_mt-1] [INFO] [1721413915.703586402] [realsense2_camera]: Open profile: stream_type: Gyro(0)Format: MOTION_XYZ32F, FPS: 200
[component_container_mt-1] [INFO] [1721413915.712259134] [realsense2_camera]: RealSense Node Is Up!

1 Like
2

Hi @hassanshafi

Thank you for your post. After running ros2 topic list, it seems that the Realsense node is not communicating with the Isaac ROS VSLAM topics.

In order to assist you better, could you please share the rqt_graph output?

Thank you in advance,
Raffaello

1 Like
3

Hello! I was able to get Visual SLAM and NVBlox working with my D456 camera. I have four D456 cameras total and need to interface them together, so it has a full 360 degree view. I see that in the most recent update on the Isaac ROS Visual SLAM: 2024-05-30 there was an added support for multi-cam VIO. Is there any documentation specifically for the multiple cam VSLAM setup with the realsense cameras? I know there is one using the Hawk cameras: Tutorial for Visual Odometry Using multiple HAWK Cameras — isaac_ros_docs documentation (nvidia-isaac-ros.github.io).

Thanks!

4

Hi @hassanshafi

Thank your reply.

At this time, the multi-cam VIO feature is postponed for a future release. It’s only available with Isaac ROS 3.0 the single-cam VIO.

At this time, we only have the demo you linked, but we are working on adding other demos in the next Isaac ROS releases.

Best,
Raffaello