Create render camera 4.5 slows simulation to standstil

Omniverse Isaac Sim
1

Hi,
I’m trying to migrate from 2023.1.1 to 4.5. When I load my robots which I used in 2023.1.1 to 4.5 are all of my omnigraphs not connected and missing half of the elements. I have now rebuilt all of them.

Now the problem is that in my camera graph which worked fine in 2023.1.1, where I create 2 render products and publish them the following happens:

In 4.5 do I have the following:

  • Both disabled: 53fps
  • As soon as I enable one of them does the simulation more or less stop and is not responsive at all (with clicking on stop a few times can I recover from this)

In 2023.1.1 did I still have a bit over 20fps with both cameras enabled.

Here the omnigraph for the camera.

image
image1641×624 97.5 KB

Why does publishing of images now all of a sudden no longer work?

Isaac Sim Version

4.5.0
2023.1.1

Operating System

Ubuntu 22.04

GPU Information

  • Model: NVIDIA GeForce RTX 3060
  • Driver Version: 550.163.01 CUDA Version: 12.4
3

Hi and welcome. Thank you for the post. I understand the fustration.

However, let me just make a couple of points.

  1. You are trying 4.5, but we have 5.0 out now, so I would first of all, use that over 4.5
  2. Your card is very very low power and well below our minimum. A 3060 barely has enough vram to even load Isaac Sim, let alone, render.
  3. Why do you need to create a render product like this? We have standard render products already built in. We have standard Movie Capture, we have the SDG, we have ROS2 output. There is no need to build a custom render product. Maybe the way you are doing it now, it simply discontinued.
  4. Finally, given the fact that you do have a very low vram card, I would strongly advise against trying to render out to TWO render products at the same time on tick. That seems like that would automatically cause massive slow downs. I would do one, and then the other.

Here is my advice. Do not use this graph at all. Just output a single standard new render product, one at a time. That will be fine.

4

Hi @Richard3D,

Thanks a lot for your reply and understanding.

  1. I’m actually still working in 2023.1.1 and trying to go to 4.5 (the reason is that I use OmniLRS (GitHub - OmniLRS/OmniLRS: Omniverse Lunar Robotics Simulator) which currently still uses 2023.1.1 but the migration to 4.5 is open for some month already.
  2. Noted, the current university computers all have max a 3060 unfortunately, I can talk to my supervisors but in the near future there is probably no hope that we get better cards.
  3. Thanks for pointing all the options out. What I basically need (and currently try to setup with the omnigraphs is:

  • Data streamed out of Isaac Sim to use in OpenVins for visual inertial odoemetry (ROS2)

    • IMU data (over 100hz)
    • 2 different camera rgb images (resolution ideally 2048x2048, I know this is high, I also tried with lower values but there was no significant increase in FPS), streamed rate in mid/high single digits fps would be enough (I tried the Simulation gate but that did still render every frame), - here I use the create render blocks (the graph which is shown in the image)
    • /tf data of the robot

  • A way to control the robot (another ROS2 graph with teleop keyboard)

  1. I understand, however in 2023.1. this still only gave me a bit more than 30fps and for the Visual Inertial Odometry pipeline do I need a stereo setup, so both camera frames as close as possible at the same time.

How would you create and output this render product as a ros2 rgb image topic?
I’m very open to any hints/resources on how I can achieve what I have outlined above.
From my understanding is the SDG and also movie capture not helpful for integrating VIO and navigation processing with Isaac sim. Especially do I not understand why what I tried to get working in 2023.1.1 now does not work at all in 4.5.

5

Hi @yhofmann! For ROS2 camera data publish, please check out the following tutorials

For TF data publish, please check out this tutorial

6

Hello!

We noticed that this topic hasn’t received any recent responses, so we are closing it for now to help keep the forum organized.

If you’re still experiencing this issue or have additional questions, please feel free to create a new topic with updated details. When doing so, we recommend mentioning or linking to this original topic in your new post—this helps provide context and makes it easier for others to assist you.

Thank you for being part of the NVIDIA Isaac Sim community.

Best regards,
The NVIDIA Isaac Sim Forum Team