FPS drops when using Camera Sensor

Note: For any Isaac Lab topics, please submit your topic to its GitHub repo (GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim) following the instructions provided on Isaac Lab’s Contributing Guidelines (Contribution Guidelines — Isaac Lab Documentation).

Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: RTX 4090

Topic Description

Detailed Description

I am using the Camera sensor impored from omni.isaac.sensor import Camera. However, as soon as I add the camera to my scene, the fps drops to around 8fps. I have tested the scene without adding any Camera, and the FPS remains at around 60 fps.

The camera is being used to capture images of boxes that spawn and then the kind of box is detected using opencv.

I add the camera using the following:

from omni.isaac.sensor import Camera

        self._camera = Camera(
            prim_path="/World/Camera",
            position=np.array([0, 0, 4]),
            resolution=(1024,720),
            orientation=euler_angles_to_quats(
                np.array([0, 90, 0]), degrees=True
            )
        )
        
        self._camera.initialize()
        self._camera.add_motion_vectors_to_frame()


Ideally, I would want my camera to be FHD, but just to get some fps out of the scene, I have reduced the camera resolution to 720p.

I understand that adding a camera means rendering another viewport in the background at camera resolution. But my GPU should be able to handle this. I have also tried destroying the main viewport so that I only have one viewport i.e. of the camera by using the following:

import omni.isaac.core.utils.viewports as vp
vp.destroy_all_viewports()

Destroying viewports doesnt give my any additional FPS.

Could someone tell me what the problem could be or if there is a better was to add a camera without dropping the fps?

Hello. Would really appreciate some help with this problem! Thank you!

Hi @saa-97! Have you tried this in Isaac Sim 4.5.0?

Hello! Yes I have tried with isaac sim 4.5 and the results are the same.

Hi @saa-97. Could you try profiler and see which process is consuming the majority of computation power? Here is the tutorial for Isaac Sim 4.5 Profiling Performance Using Tracy — Isaac Sim Documentation

Hi there,

I could not repro the large FPS drop on my side in 4.5. Could it be that other processing on the data cold cause this?

In the warehouse scenario I get ~95 FPS, adding the camera (1024x720) with the almost same resolution as the viewport (1280x720) gets get me to ~50 FPS, which is expected by doubling the rendering efforts.

You can also disable the viewport rendering if not needed using:

from omni.kit.viewport.utility import get_active_viewport
get_active_viewport().set_updates_enabled(False)