[2022.2.1] Rendering 20+ cameras without running out of memory

Omniverse Isaac Sim
1

Hi!

I’m interested in simulating more than +20 cameras for RL / collecting image observations with Omniverse Isaac Gym. With 2022.2.1, the max cameras I’m able to simulate on my RTX 4090 is around 20 cameras before running out of GPU memory. This is an empty scene with simple cube, running in headless mode.

  • When do we expect camera related optimizations’ to hit Isaac sim?
  • Currently it seems that the memory cost per camera is somewhat fixed. I. even if the camera renders at 64x64 pixels, or 1024x1024, both cases utilize similar GPU footprint. And in both cases i get out of memory exception when going over 20 cameras.
  • Are there any workarounds in the interim? Tiled rendering? Disabling all Raytracing effects? (is that possible?)

For testing I’m using the camera.py in the standalone_examples sensor folder, but instead of instantiating one camera, i instantiate 20 camera in a loop on start.

3

Hi there, we are actively working on improving the camera support in Isaac Sim to be able to support more cameras. We will be releasing a new RL example with cameras as part of the OmniIsaacGymEnvs repo soon, which will provide some settings on improving the performance. However, more improvements are in the works and will continue to be release in the future.

4

That’s great news, thanks for the update!