Some question about isaac sim and vslam

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Test environment
HW:
CPU : Intel(R) Xeon(R) w5-3435X
GPU : RTX-4090
vslam carter : from Tutorial for Visual SLAM with Isaac Sim — isaac_ros_docs documentation
nvblox carter : from Isaac Sim Examples — isaac_ros_docs documentation
SW : Isaac sim 4.5 + Isaac vslam 3.2

Test Step.

1. Load sample scene which as description in vslam isaac sim example.

2. launch isaac_ros_visual_slam_isaac_sim.launch.py as tutorial and press Play in isaac sim

3. monitor the publish rate of clock and compare to real time.

4. monitor the publish rate of /visual_slam/tracking/vo_pose

5. Remove the nova carter in scene and add another nova carter which copy from nvblox example to do the same test above

Questions
(1)
when run vslam scene + vslam carter, we monitor the different between sim time and real time.
sim time 1s => real time 2.3s
Is these expected?
If not, what can I check to make it more sync between real time and sim time or it is our HW limitation.

(2) test vslam carter (1920x1200 resolution)
In vslam carter, when enable second camera(like right camera, just check enabled),
it seems the image publish rate is lower than expected.(like right/image is half of front/image)
what can I check to make sure the publish rate is normal(normal means front and right can have similar rate)

(3) test nvblox carter
When enable multi camera in nvblox carter running in vslam scene,
I monitor update rate of /visual_slam/tracking/vo_pose, the update rate seems become half when using 2 cameras compare to 1 camera.
Is this expected?
If not, what can I check?

2

Hi chengcheng,

(1) Yes, in general it’s normal for sim time to lag behind real-time. You may refer to this performance handbook to check for further optimization.

(2) Please check this tutorial to see if you set frameSkipCount to have such a publish rate difference. Also check if the camera resolutions are different (tutorial).

(3) Yes, that is expected when you have multiple cameras running. But sim time is different from real time. How did you monitor the publish rate? Did you do it by using ros2_topic_hz or by checking the timestamp in the header? ros2 topic hz is based on wall time if you did not set use_sim_time to true.

Additionally you may try troubleshoot with factory settings.

3

Hi William,

Thanks for response.

As for (3), I use a python script which will subscribe the topic and check with real time and sim time.

The output is like below

(a) one camera

[INFO] [1761543710.325930128] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 58.00 Hz (sim_time = 6.000s)
[INFO] [1761543714.416051852] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 55.00 Hz (sim_time = 7.000s)
[INFO] [1761543718.497856991] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 52.00 Hz (sim_time = 8.000s)
[INFO] [1761543722.602302578] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 55.00 Hz (sim_time = 9.000s)
[INFO] [1761543726.699055059] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 52.00 Hz (sim_time = 10.000s)

(b) two camera

[INFO] [1761543590.213153069] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 36.61 Hz (sim_time = 6.000s)
[INFO] [1761543594.455075965] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 39.68 Hz (sim_time = 7.000s)
[INFO] [1761543598.541919409] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 36.00 Hz (sim_time = 8.000s)
[INFO] [1761543602.635702693] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 37.24 Hz (sim_time = 9.000s)
[INFO] [1761543606.783887778] [topic_rate_monitor]: [/visual_slam/tracking/vo_pose] avg_freq = 36.77 Hz (sim_time = 10.000s)

The rate is calculated based on sim time.

From the log, 1 second in sim time will be around 4 second in real time.

Is the rate of vo_pose like above between 1 and 2 camera expected (from 5xHz to 3xHz after use 2 cameras for vslam)?

BTW, to control the output rate of camera, just change frameSkipCount is enough, right?

In the tutorial, it seems mentioned ActionGraph, but I don’t see anywhere use it in our testing environment. Do I need to setup some ActionGraph?

As for (2), it can output as expect now.

But when I try to run 2 camera with this 1920x1200 resolution. It seems will affect the frame rate of image topic and cause vslam can’t work well

Here is the example of output frame rate

INFO] [1761545767.781951532] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 50.16 Hz (sim_time = 236.000s)
[INFO] [1761545771.362406785] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 55.00 Hz (sim_time = 237.000s)
[INFO] [1761545774.908675041] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 52.00 Hz (sim_time = 238.000s)
[INFO] [1761545778.506122927] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 53.00 Hz (sim_time = 239.000s)
[INFO] [1761545782.413864391] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 24.00 Hz (sim_time = 240.000s)
[INFO] [1761545786.352308045] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 14.48 Hz (sim_time = 241.000s)
[INFO] [1761545790.315919190] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 23.23 Hz (sim_time = 242.000s)
[INFO] [1761545794.360182318] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 14.48 Hz (sim_time = 243.000s)
[INFO] [1761545798.337237468] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 16.84 Hz (sim_time = 244.000s)
[INFO] [1761545802.294282740] [topic_rate_monitor]: [/front_stereo_camera/left/image_rect_color] avg_freq = 13.85 Hz (sim_time = 245.000s)

Before I start vslam node, the output frame rate will around 5x Hz. After start vslam node, it drop to 1x ~ 2x Hz. Is this caused by computing resource or something else?

Thanks for help.

4

Hi chengcheng,

Although not exactly your case, there are two benchmarks that we published you could refer to:

  1. vslam could reach 30fps with 4 hawk cameras turned on and without Isaac Sim, as benchmarked here. (does not run rendering)

  2. Additionally, the Isaac Sim benchmark gives a good reference on the frame rate of running Nova Carter ROS2 bridge example (4 hawk cameras) on Isaac Sim. (does not run nvblox or vslam)