How can we get simulation time close to real time?

I’m setting up a scene with 3 Nova Carters running ROS2 (I need the Nav2 Stack to send waypoints). I’m trying now to get it as close to real-time as possible for demonstration and data generation purposes. However, the best I’ve been able to get is a Real-Time-Factor of about 0.14 (a single robot gets an RTF of 0.19).

The RTF started at about .08 until I disabled nearly every sensor’s action graph/node but the right front hawk camera, the 3D lidar, /odom, and /cmd_vel. These are about the bare minimum for running a nav2 stack and getting the robots to move around autonomously. I also am skipping every other frame for publishing the other sensors.

Some other data points:

  • No robot: 0.9 RTF
  • Nova Carter Sensors: 0.88 RTF
  • Single Nova Carter ROS2: 0.19 RTF
  • 3 Nova Carter ROS2: 0.14 RTF

If I disable rendering any cameras or lidars, Ican get up to about 0.5 RTF. But then the simulation isn’t very valuable and I can’t run nav2.

I’m using the AWS Nvidia AMI with a g5.8xlarge computer. Here’s the output of nvidia-smi:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A10G                    Off | 00000000:00:1E.0 Off |                    0 |
|  0%   42C    P0              99W / 300W |   6457MiB / 23028MiB |     52%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A    412649    C+G   /isaac-sim/kit/kit                         6415MiB |
+---------------------------------------------------------------------------------------+

Although I did try upgrading it to a computer with a handful of very powerful GPUs and didn’t really move the needle much.

I’m thinking there are other settings or factors I can tweak here? Probably something buried in the settings, like a render rate or physics step I can play with?

Please refer to our speedup cheat sheet, there may be a few tweaks you can benefit from.

Exactly what I’m looking for, thanks! Will play with this soon and report back

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.