When IMU mode is turned on, I found that the topic /visual_slam/vis/gravity sometimes appears and sometimes disappears in RVIZ2 (as shown in the picture).
I have a question, when does the IMU fusion mode of the latest version of cuvslam3.1 use IMU data? Is it to use IMU integration to make up when there are fewer feature points? Or is the IMU data added to the backend optimization process and processed together with the image feature points?
From the gravity acceleration publishing section in the visual_slam_impl.cpp file of the ROS2 source code (as shown in the screenshot below), we know that CUVSLAM starts aligning camera data and IMU data and calculating and publishing the gravity acceleration vector after it is enabled or when visual tracking is lost, right? However, in actual operation with IMU mode enabled, it was found that IMU data initialization and gravity acceleration calculation only occur when visual features are lost, is that correct?
Hello @lmljll,
cuVSLAM calculates and publishes gravity vector after camera-to-IMU alignment, which occurs at startup or after loss of visual tracking.
After startup (when the system initializes, but only after sufficient keyframes are collected and aligned to estimate the gravity vector)
After loss of visual tracking (when visual features are lost and the system must realign on recovery).
If you see gravity vector publishing only after a visual loss, it is because that’s when alignment and gravity estimation are (re)triggered.
Thank you very much. Could you please tell me how many keyframes are typically needed to estimate the gravity vector, or how long does the initialization process take?
Not sure exactly how many keyframes would be enough but you can try moving the camera calm and smooth at the beginning (sideways, down) to get enough more diverse feature to reach keyframes. After ~10-20 sec of movements, gravity vector should appear as a huge gray gravity arrow pointing down if you set ‘enable_imu_fusion’: True.
