During the normal operation of cuvslam, the IMU input queue may be empty. After checking that it is not a ROS2 input problem, I suspect that the reason seems to be that when cuvslam takes a long time to run, it blocks the input of the synchronizer IMU data, causing the synchronizer to delete the cached IMU data as outdated messages. The following image shows the time taken by cuvslam each time:
The above are the log records before and after the IMU queue is empty. The IMU message interval is 5ms and the image interval is 33ms. However, judging from the ros2 log, they are both delayed, causing the IMU data to be deleted by the synchronizer. Questions:
In IMU-enabled mode, if the IMU input is empty, does cuvslam still use IMU information when calculating poses? Or does it switch to image-only mode?
If cuvslam takes a long time, and then the IMU data is empty, how can I resolve this issue?
Hello, @Ahung thank you for your reply.
My environment configuration is as follows:
Hardware: Jetson Orin Nano 8GB
OS: Jetpack 6.2
Version: Isaac ROS V3.2.0
Startup parameters correspond to the following:
It looks that you modified many parameters for your sensor and added some prints in visual_slam_node. Could you provide the changes you made so that we can try to reproduce it?
Hello，@Ahung
I didn’t make any major changes to the code. I simply printed out in the UpdatePose section whether and how much IMU data cuvslam used. This was to clarify why the terminal displayed IMU data loss each time the node was executed. The printed message showed that the synchronizer output IMU data was empty every time cuvslam took a long time to execute. After reviewing the synchronizer code, I personally believe that the callback and data matching within the synchronizer are running on the same thread. When the callback (that is, UpdatePose) executes for a long time, it blocks the data input, causing the IMU to be deleted as old data. As a verification, you can annotate the tasks within the UpdatePose function with a manual 30ms delay and print the number of IMUs input to the function each time. You will see that the IMU data is empty in more frames.
The image above is the log printed by the terminal. The IMU input of cuvslam is empty for three consecutive frames. The yellow annotation is the timestamp of the IMU callback input during these three frames. The interval has exceeded the default 5ms.
IMU message drop is expected behavior if:
a) Messages in the buffer are out of date relative to the time interval between the previous and current frames
b) Messages in the buffer are not sequential
Longer processing of certain rare frames (keyframes) is expected behavior of cuVSLAM
If cuVSLAM in IMU fusion mode does not receive valid IMU measurements for a sequence of image pairs, it switches to simple VO stereo tracking mode
Conclusion: The ability to drop values from the IMU buffer (if messages in it are out of date relative to the current pair) is expected behavior.
Other notes:
The user is using Jetson Orin Nano, which is a relatively weak platform (the target platform for SQA testing of all cuVSLAM modes was Orin AGX). We do not expect stable operation in modes requiring intensive real-time information processing. For Orin Nano, we recommend using VO Stereo Tracking (for greater stability try 60 FPS and VGA resolution) if Orin Nano can handle it.
The user is using IMU message buffers of 5000 messages (IMU message history for 25 seconds at 200 Hz) and 100 picture messages (picture history for 3.3 seconds if the camera is in 30 FPS mode):
Buffers of this size are generally unusual for a system operating in real time, as they attempt to work with information received several seconds ago.
We do not recommend using such long buffers. If the system cannot handle real-time information processing, overflow and forced clearing of messages in the buffer will lead to jumps in the information coming from the buffers to cuVSLAM
IMU messages are valid only within a pair of consecutive messages (see 1a), so a balance between the buffer values is necessary
The user is using a jitter threshold = 300ms for images, which corresponds to a 3 FPS camera. We expect stable stereo tracking for 30 FPS and higher
The user is using a maximum path length of 8000 poses. The path is used only for visualization, which can affect the performance of the ROS Node (due to the processing of such a long vector). We recommend reducing this value to 1, as this parameter does not affect the operation of the core library of cuVSLAM. The size of the SLAM map used by cuVSLAM is determined by the parameter slam_max_map_size = 300. This map size is determined experimentally, including within tests on large warehouses
