CuvSLAM IMU fusion is not improving performance

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hi,

I am currently using release 3.2 and running nvblox with cuvslam. I noticed that the vslam performance is degraded when facing object with less feature (e.g. white wall). I tried to enable the imu fusion to improve the performance, but I am not seeing much differences (when facing the featureless area and do a rotation, the slam pose with or without imu are very similar, and an obvious drift could be observed). I can verify that the imu is providing correct readings and the visual_slam_node is subscribing to the topic. But I do see some of this warning, which I am not sure how it will impact the overall performance:

component_container_mt-3] [WARNING] Lost IMU msgs: 2, Frame time delta = 0.027793 [s], drop ratio = 40.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.021909 [s], drop ratio = 25.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 3, Frame time delta = 0.022254 [s], drop ratio = 75.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.018615 [s], drop ratio = 33.333336 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.022253 [s], drop ratio = 25.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 2, Frame time delta = 0.022256 [s], drop ratio = 50.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 4, Frame time delta = 0.022254 [s], drop ratio = 100.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.022256 [s], drop ratio = 25.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.022825 [s], drop ratio = 25.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.022445 [s], drop ratio = 25.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 4, Frame time delta = 0.022254 [s], drop ratio = 100.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 2, Frame time delta = 0.022257 [s], drop ratio = 50.000000 [%]
[component_container_mt-3] [WARNING] Lost IMU msgs: 1, Frame time delta = 0.025692 [s], drop ratio = 25.000000 [%]

Could you please let me know if integrating IMU can help solve the low-feature area drift error? Or if I should integrate wheel odometry (if yes, how can I do this)?

2

Hi @lancexiao96

Thank you for your post.

IMU fusion does not improve the quality of stereo tracking and can only correct short-term visual artifacts (such as blurring due to camera movement or exposure changes). For Isaac ROS 3.2, the following options are available:

  • The best option to handle featureless visual input using cuVSLAM is a multicamera mode (live camera ROS2 example with Hawk cameras)

  • The Isaac Perceptor stack provides an example of replacing cuVSLAM with wheel+IMU odometry

If the goal is to run NVBLOX + cuVSLAM, you may want to consider the Python bindings:

  • NVBLOX reconstruction could work in Python with Nvblox-torch + PyCuVSLAM (live Realsense example)

  • Live multicamera PyCuVSLAM example with Realsense cameras

  • If only 1 camera is available, consider RGB-D mode (only in PyCuVSLAM, live camera examples 1, 2), more stable operation than VIO is expected

Best,
Ahung

3

Hi,

Thank you very much for your reply.

My goal is to use NVBLOX + cuVSLAM for mapping and localization, then use nav2 to control the real robot to navigate within the map. Therefore, I would prefer to use Isaac ROS.

Regarding the multi-camera setup, my system currently has 3 realsense d435i for cuvSLAM and NVBLOX. However, based on the performance and the comment in the code base, I think 3.2 only uses one camera for cuVSLAM when combined with nvblox, and the rest of the cameras are only used in nvblox reconstruction but not for slam. The comment in the code is below:
”# NOTE(…, 19.08.2024): At the moment in nvblox_examples we only support a single camera running cuVSLAM, even in the multi-camera case: we run nvblox on multiple cameras, but cuVSLAM on camera0 only.”

I am wondering if a realsense multi-camera vslam combined with nvblox is available in ros or is it only available with python bindings?

4

Hi @lancexiao96

Currently, the realsense multi-camera support for NVBLOX + cuVSLAM is only available in python bindings.

Best,
Ahung

5

Hi,

Thank you for the reply.

May I ask why is the multi-camera setup not supported in ros? Also, is this something that will be considered adding?