ISAAC_ROS_VSLAM: No messages being published

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

I am facing an issue where I am unable to get any messages from ros2 topic echo /visual_slam/tracking/odometry. (Or any other /visual_slam/tracking/* topics). This is strange because I am able to see publisher count = 1 but no messages are actually being published. I would appreciate any help or insights on how to resolve this issue so that I can get messages to be properly published.

Background

I am using Jetson Orin Nano 8gb Developer Kit. I have followed this guide step by step and have flashed the provided custom SD card for Jetson Orin Nano. I am using Intel Realsense D455, which is a compatible model, and I can see the camera images when I run realsense-viewer. I have also updated the firmware to 05.15.01 which meets the criteria “firmware of the RealSense camera is 5.14.0 or newer.”

Output of ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_realsense.launch.py:

[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [78195]
[INFO] [realsense2_camera_node-2]: process started with pid [78197]
[component_container-1] [INFO] [1708984640.099437692] [visual_slam_launch_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_visual_slam/lib/libvisual_slam_node.so
[realsense2_camera_node-2] [INFO] [1708984640.478058622] [camera.camera]: RealSense ROS v4.51.1
[realsense2_camera_node-2] [INFO] [1708984640.478491693] [camera.camera]: Built with LibRealSense v2.53.1
[realsense2_camera_node-2] [INFO] [1708984640.478531055] [camera.camera]: Running with LibRealSense v2.53.1
[component_container-1] [INFO] [1708984640.502260655] [visual_slam_launch_container]: Found class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-1] [INFO] [1708984640.511121286] [visual_slam_launch_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-1] [INFO] [1708984640.689291824] [visual_slam_node]: Elbrus version: 10.5
[realsense2_camera_node-2] [INFO] [1708984640.691532511] [camera.camera]: Device with serial number 038122250330 was found.
[realsense2_camera_node-2] 
[realsense2_camera_node-2] [INFO] [1708984640.699417427] [camera.camera]: Device with physical ID /sys/devices/platform/3610000.xhci/usb1/1-2/1-2.3/1-2.3:1.0/video4linux/video0 was found.
[realsense2_camera_node-2] [INFO] [1708984640.701959884] [camera.camera]: Device with name Intel RealSense D455 was found.
[realsense2_camera_node-2] [INFO] [1708984640.704450020] [camera.camera]: Device with port number 1-2.3 was found.
[realsense2_camera_node-2] [INFO] [1708984640.707527920] [camera.camera]: Device USB type: 2.1
[realsense2_camera_node-2] [WARN] [1708984640.709148168] [camera.camera]: Device 038122250330 is connected using a 2.1 port. Reduced performance is expected.
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/visual_slam_node' in container '/visual_slam_launch_container'
[realsense2_camera_node-2] [INFO] [1708984640.762462647] [camera.camera]: getParameters...
[realsense2_camera_node-2] [INFO] [1708984640.768639599] [camera.camera]: JSON file is not provided
[realsense2_camera_node-2] [INFO] [1708984640.768722962] [camera.camera]: Device Name: Intel RealSense D455
[realsense2_camera_node-2] [INFO] [1708984640.768748947] [camera.camera]: Device Serial No: 038122250330
[realsense2_camera_node-2] [INFO] [1708984640.768771412] [camera.camera]: Device physical port: /sys/devices/platform/3610000.xhci/usb1/1-2/1-2.3/1-2.3:1.0/video4linux/video0
[realsense2_camera_node-2] [INFO] [1708984640.768789589] [camera.camera]: Device FW version: 05.15.01.00
[realsense2_camera_node-2] [INFO] [1708984640.768806645] [camera.camera]: Device Product ID: 0x0B5C
[realsense2_camera_node-2] [INFO] [1708984640.768819926] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-2] [ERROR] [1708984640.928006150] [camera.camera]: Given value, 640x360x90 is invalid. Set ROS param back to: 640x480x15
[realsense2_camera_node-2] [WARN] [1708984640.946156738] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range.
[realsense2_camera_node-2] [INFO] [1708984641.062888577] [camera.camera]: Stopping Sensor: Stereo Module
[realsense2_camera_node-2] [INFO] [1708984641.121916120] [camera.camera]: Starting Sensor: Stereo Module
[realsense2_camera_node-2] [INFO] [1708984641.310861468] [camera.camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-2] [INFO] [1708984641.311643864] [camera.camera]: Stopping Sensor: Motion Module
[realsense2_camera_node-2] [INFO] [1708984641.348215515] [camera.camera]: Starting Sensor: Motion Module
[realsense2_camera_node-2] [INFO] [1708984641.396753089] [camera.camera]: Open profile: stream_type: Accel(0)Format: MOTION_XYZ32F, FPS: 63
[realsense2_camera_node-2] [INFO] [1708984641.400138968] [camera.camera]: Open profile: stream_type: Gyro(0)Format: MOTION_XYZ32F, FPS: 200
[realsense2_camera_node-2] [INFO] [1708984641.416299695] [camera.camera]: RealSense Node Is Up!
[realsense2_camera_node-2] [WARN] [1708984641.512268630] [camera.camera]:

(The only red flag I see is Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range., but I am not sure if this is the core issue)

Output of ros2 wtf --report:

topic [type]            : /visual_slam/tracking/odometry [nav_msgs/msg/Odometry]
publisher node          : visual_slam_node
subscriber node         : _ros2cli_13486
compatibility status    : OK
...
topic               : /visual_slam/tracking/odometry
publisher count     : 1
subscriber count    : 0
topic               : /visual_slam/tracking/slam_path
publisher count     : 1
subscriber count    : 0
...

Output of ros2 node list

/camera/camera
/launch_ros_78184
/transform_listener_impl_aaaacb182750
/visual_slam_launch_container
/visual_slam_node

Additional information:

  • I have tried stopping and starting the daemon
  • I tried recreating the Docker container (exiting and running isaac_ros_container again)
  • Power-mode is set to 15W for the Jetson
  • I can read from /camera/* topics, but the average rate is a little slow: ros2 topic hz /camera/infra1/image_rect_raw --window 20 average rate: 12.309 min: 0.055s max: 0.200s std dev: 0.03933s window: 13
  • I’ve also tried removing /build /log /install and tried colcon build --symlink-install but the issue persists.
2

Hi @brian79891

The demo you are following may be outdated, which could be causing the errors you are seeing.

I suggest following more updated demo on Isaac ROS VSLAM:

Let me know if I helped you.

Best,
Raffaello

3

Was this issue solved? I am facing a similar issue. I am providing the right topics, and the images and info are getting published on those topics, but Slam doesn’t publish any points on any topic. What could be the reason? I am using two RGB cameras with a 48-mm baseline in Gazebo.

4

Hi @ssharma4

Thank you for joining the Isaac ROS forum.

Can you explain your environment setup?
Which demo are you testing?

Best,
Raffaello

5

The following is the github link that describes the environment setup:
Github issue

I am using the following tutorials:
Link

After joining office hours today, I changed the following parameters to true, but still no point clouds are getting published for visualization. I also did colcon clean workspace and colcon build --symlink-install after the changes :

enable_observations_view

bool

false

If enabled, 2D feature pointcloud will be available for visualization.

enable_landmarks_view

bool

false

If enabled, landmark pointcloud will be available for visualization.

enable_slam_visualization

bool

false

Main flag to enable or disable visualization.

I have also changed the frame names in the slam code for left camera, right camera, and base_link to the correct names.

I am also confused about how CuVSLAM gets the baseline of the stereo since there is no config file. Should I publish a transform from left camera to right camera to give the slam a baseline? I saw that the Slam subscribes to TF2 transformations. Also, since there was no map frame in the simulation, I created a static transform from map to odom just in case Slams wants that frame to get a global reference.

edit1:

I am able to see observation points after I remove the remapping in the launch file code:

       # remappings=[('/stereo_camera/left/camera_info', '/camera_info_left'),
        #             ('/stereo_camera/right/camera_info', '/camera_info_right')],

The info topic is incorrectly mentioned in the documentation.

I don’t see the landmark points because the messages are there, but there is no data in those messages, and the tracking is lost almost all the time.

See the screenshot:

Screenshot from 2024-05-29 13-49-44
Screenshot from 2024-05-29 13-49-441621×864 69.3 KB

The camera frame and baselink frame are drifting in rviz even though the robot is not moving, which is to be expected since tracking is lost.

The output on the cuVSLAM terminal:

[component_container-1] [WARN] [1717005539.372907907] [visual_slam_node]: Visual tracking is lost
[component_container-1] [WARN] [1717005539.402140555] [visual_slam_node]: Delta between current and previous frame [67.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.435181309] [visual_slam_node]: Delta between current and previous frame [266.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.438879992] [visual_slam_node]: Visual tracking is lost
[component_container-1] [WARN] [1717005539.472957636] [visual_slam_node]: Delta between current and previous frame [68.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.505535118] [visual_slam_node]: Delta between current and previous frame [102.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.545626781] [visual_slam_node]: Delta between current and previous frame [165.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.686052825] [visual_slam_node]: Delta between current and previous frame [234.000000] is above threshold [33.330000]
[component_container-1] [WARN] [1717005539.686896993] [visual_slam_node]: Visual tracking is lost
[component_container-1] [WARN] [1717005539.747173277] [visual_slam_node]: Delta between current and previous frame [64.000000] is above threshold [33.330000]

Your help in getting this working will be greatly appreciated.

6

Hi @ssharma4

Have you tried the new Isaac ROS 3.0? We have made major updates to Isaac ROS VSLAM and may be able to fix your issue.

Let me know

Best,
Raffaello

7

Hi Raffaello,

I am using the latest Isaac cuVSLAM docker container. I am still not getting any points published on any topic.
The following are the frame rate and topics:

shivam157@ubuntu:~/dai_ws$ ros2 topic hz /visual_slam/image_0
average rate: 8.518
	min: 0.021s max: 0.738s std dev: 0.21996s window: 9
average rate: 8.173
	min: 0.021s max: 0.790s std dev: 0.23480s window: 17
average rate: 7.018
	min: 0.021s max: 0.790s std dev: 0.24933s window: 22
^Cshivam157@ubuntu:~/dai_ws$ ros2 topic hz /visual_slam/image_1
average rate: 8.261
	min: 0.031s max: 0.776s std dev: 0.23181s window: 9
average rate: 7.103
	min: 0.031s max: 0.858s std dev: 0.26581s window: 15
average rate: 6.626
	min: 0.030s max: 0.858s std dev: 0.26984s window: 21
shivam157@ubuntu:~/dai_ws$ ros2 topic hz /visual_slam/camera_info_0
average rate: 33.645
	min: 0.001s max: 0.053s std dev: 0.01393s window: 35
average rate: 34.183
	min: 0.001s max: 0.054s std dev: 0.01426s window: 70
average rate: 33.571
	min: 0.001s max: 0.063s std dev: 0.01456s window: 104
average rate: 33.629
	min: 0.001s max: 0.063s std dev: 0.01404s window: 138
^Cshivam157@ubuntu:~/dai_ws$ ros2 topic hz /visual_slam/camera_info_1
average rate: 29.330
	min: 0.011s max: 0.057s std dev: 0.01039s window: 31
average rate: 30.975
	min: 0.002s max: 0.062s std dev: 0.01267s window: 64
average rate: 31.467
	min: 0.001s max: 0.066s std dev: 0.01363s window: 98
^Cshivam157@ubuntu:~/dai_ws$

The above output is when I am running cuVSLAM on another terminal. When I don’t run slam and only run the ROS camera driver for my custom stereo camera I get 30 fps on the both of the images. So what is the slam software doing that is dropping the fps on the images? I am using jetson orin nano 8 gb developer kit.

Please help me with this issue.