Hi i have a problem that I’ve been having for a long time .
I have been using Isaac ros VSLAM with d435i on my Jetson Orin Nano Devkit and I’ve had a problem where the vslam works fine until it sometimes losses tracking (I guess) and the whole odometry pose moves drastrictly to some very random position , In my nav2 applications with nvblox or something like slam_toolbox it results in the whole map moving somewhere and being recreated ,
Ive recorded a video with the default settings on the quickstart page for vslam where i move it around for sometime and then cover the sensor which results in the very weird pose jumps , after that i uncover it and it continues making new points from that position .
If you know maybe just how to maybe filter it or something like that which would be sufficient i would appreciate it very much.
Thanks for any help in advance! :)
Hi Iam really sorry for the late response here is the full log and when i cover the sensor (which replicates the problem that happens sometimes when using vslam for my applications) the message is visual tracking is lost
admin@ubuntu:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_realsense.launch.py
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2025-02-11-10-05-11-020208-ubuntu-3875
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [3887]
[INFO] [realsense2_camera_node-2]: process started with pid [3889]
[realsense2_camera_node-2] [INFO] [1739264711.407675255] [camera.camera]: RealSense ROS v4.51.1
[realsense2_camera_node-2] [INFO] [1739264711.407859578] [camera.camera]: Built with LibRealSense v2.55.1
[realsense2_camera_node-2] [INFO] [1739264711.407883726] [camera.camera]: Running with LibRealSense v2.55.1
[component_container-1] [INFO] [1739264711.453652536] [visual_slam_launch_container]: Load Library: /opt/ros/humble/lib/libvisual_slam_node.so
[component_container-1] [INFO] [1739264711.543484615] [visual_slam_launch_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::visual_slam::VisualSlamNode
[component_container-1] [INFO] [1739264711.543638937] [visual_slam_launch_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::visual_slam::VisualSlamNode
[component_container-1] [INFO] [1739264711.561674566] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container-1] [INFO] [1739264711.563659360] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_gxf_helpers.so
[component_container-1] [INFO] [1739264711.569264379] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sight.so
[component_container-1] [INFO] [1739264711.575490207] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_atlas.so
[component_container-1] 2025-02-11 10:05:11.589 WARN external/com_nvidia_gxf/gxf/std/program.cpp@538: No GXF scheduler specified.
[component_container-1] [INFO] [1739264711.589984577] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container-1] [INFO] [1739264711.598507201] [visual_slam_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container-1] [INFO] [1739264711.600244503] [visual_slam_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container-1] [INFO] [1739264711.621648037] [visual_slam_node]: cuVSLAM version: 12.6
[component_container-1] [INFO] [1739264711.715184907] [visual_slam_node]: Time taken by CUVSLAM_WarmUpGPU(): 0.093460
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node ‘/visual_slam_node’ in container ‘/visual_slam_launch_container’
[realsense2_camera_node-2] [INFO] [1739264712.074125391] [camera.camera]: Device with serial number 044322070335 was found.
[realsense2_camera_node-2]
[realsense2_camera_node-2] [INFO] [1739264712.074318830] [camera.camera]: Device with physical ID 2-1.3-3 was found.
[realsense2_camera_node-2] [INFO] [1739264712.074353373] [camera.camera]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-2] [INFO] [1739264712.075058330] [camera.camera]: Device with port number 2-1.3 was found.
[realsense2_camera_node-2] [INFO] [1739264712.075105858] [camera.camera]: Device USB type: 3.2
[realsense2_camera_node-2] [INFO] [1739264712.085180950] [camera.camera]: getParameters…
[realsense2_camera_node-2] [INFO] [1739264712.087076220] [camera.camera]: JSON file is not provided
[realsense2_camera_node-2] [INFO] [1739264712.087142395] [camera.camera]: Device Name: Intel RealSense D435I
[realsense2_camera_node-2] [INFO] [1739264712.087161393] [camera.camera]: Device Serial No: 044322070335
[realsense2_camera_node-2] [INFO] [1739264712.087173835] [camera.camera]: Device physical port: 2-1.3-3
[realsense2_camera_node-2] [INFO] [1739264712.087183014] [camera.camera]: Device FW version: 5.13.0.50
[realsense2_camera_node-2] [INFO] [1739264712.087192098] [camera.camera]: Device Product ID: 0x0B3A
[realsense2_camera_node-2] [INFO] [1739264712.087200701] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-2] [WARN] [1739264716.289073691] [camera.camera]: re-enable the stream for the change to take effect.
[realsense2_camera_node-2] [ERROR] [1739264717.537611538] [camera.camera]: Given value, 200 is invalid. Set ROS param back to: 63
[realsense2_camera_node-2] [INFO] [1739264717.554543986] [camera.camera]: Stopping Sensor: Stereo Module
[realsense2_camera_node-2] [INFO] [1739264717.895665187] [camera.camera]: Starting Sensor: Stereo Module
[realsense2_camera_node-2] [INFO] [1739264717.989365959] [camera.camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 640, Height: 360, FPS: 90
[realsense2_camera_node-2] [INFO] [1739264717.989589175] [camera.camera]: Open profile: stream_type: Infra(2), Format: Y8, Width: 640, Height: 360, FPS: 90
[realsense2_camera_node-2] [INFO] [1739264717.989703773] [camera.camera]: Stopping Sensor: Motion Module
[realsense2_camera_node-2] [INFO] [1739264718.027750026] [camera.camera]: Starting Sensor: Motion Module
[realsense2_camera_node-2] [INFO] [1739264718.051957700] [camera.camera]: Open profile: stream_type: Accel(0)Format: MOTION_XYZ32F, FPS: 63
[realsense2_camera_node-2] [INFO] [1739264718.052104634] [camera.camera]: Open profile: stream_type: Gyro(0)Format: MOTION_XYZ32F, FPS: 200
[realsense2_camera_node-2] [INFO] [1739264718.055397859] [camera.camera]: RealSense Node Is Up!
[realsense2_camera_node-2] [WARN] [1739264718.065312683] [camera.camera]:
[component_container-1] [INFO] [1739264718.428882660] [visual_slam_node]: Initializing cuVSLAM.
[component_container-1] [INFO] [1739264718.430728068] [visual_slam_node]: Use use_gpu: true
[component_container-1] [INFO] [1739264718.431785265] [visual_slam_node]: Enable IMU Fusion: true
[component_container-1] [INFO] [1739264718.431817441] [visual_slam_node]: gyroscope_noise_density: 0.000244
[component_container-1] [INFO] [1739264718.431839030] [visual_slam_node]: gyroscope_random_walk: 0.000019
[component_container-1] [INFO] [1739264718.431849712] [visual_slam_node]: accelerometer_noise_density: 0.001862
[component_container-1] [INFO] [1739264718.431858668] [visual_slam_node]: accelerometer_random_walk: 0.003000
[component_container-1] [INFO] [1739264718.431867112] [visual_slam_node]: frequency: 200.000000
[component_container-1] [INFO] [1739264723.600805609] [visual_slam_node]: Time taken by CUVSLAM_CreateTracker(): 5.168927
[component_container-1] [INFO] [1739264723.601756619] [visual_slam_node]: cuVSLAM tracker was successfully initialized.
[component_container-1] [WARN] [1739264723.967185339] [visual_slam_node]: Delta between current and previous frame [365.925888 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264736.171563718] [visual_slam_node]: Visual tracking is lost
Dont know if its related but sometimes i get this “error”/message also
[component_container-1] [WARN] [1739264803.056086054] [visual_slam_node]: Delta between current and previous frame [22.246144 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.134909138] [visual_slam_node]: Delta between current and previous frame [22.246912 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.200924399] [visual_slam_node]: Delta between current and previous frame [22.246400 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.267009117] [visual_slam_node]: Delta between current and previous frame [22.245632 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.345062703] [visual_slam_node]: Delta between current and previous frame [22.245120 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.400805626] [visual_slam_node]: Delta between current and previous frame [22.243584 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.467498532] [visual_slam_node]: Delta between current and previous frame [22.243072 ms] is above threshold [22.000000 ms]
[component_container-1] [WARN] [1739264803.523578118] [visual_slam_node]: Delta between current and previous frame [22.243840 ms] is above threshold [22.000000 ms]
I also encountered the same problem. I guess it might be the large jump error caused by using IMU integration and motion model calculation after visual feature tracking fails.
This issue you can solve by increasing the frame rate of your camera that vslam uses. When you launch the file, at some point you will see something like:
So you need to find the code block that sends the fps param into the relasense camera node and change it. You can first use realsense-viewer to figure out the different resolutions and fps of your camera and then change the launch file accordingly.
Regarding your first question, it’s probably an imu issue or frame drop issue but I’m not sure.
The fps of the camera as it’s to do with “current and previous frame” so the infra 1, infra 2, colour. You can of course also change the gyro and the accel fps. Check the launch file which defines the params. There isn’t a comprehensive document anywhere that lists all the vslam params unfortunately… perhaps someone from NVIDIA can look into that.
I got the same problem on a ZED X running isaac vslam or the its provided stereolabs vslam, when tracking is lost the pose gets shifted a lot , will try to mitigate it using an ekf with some other sensors , How did you solve it?
My idea is the same as yours. We may add external odom through ekf to alleviate this problem in the future. But the bad thing is that introducing external odom can solve the pose jitter but may not be able to repair the vslam map data.The localization module may not be able to use the cuvslam map data.If you have a better way to solve this problem, please let me know. Thank you!