The screen freezes while running Isaac-Sim 4.2.0

Omniverse Isaac Sim
,
1

We are trying to add a panel (like a machine info table) in the viewport of isaac-sim and update some values on that panel.

We had tried some extensions that can make this work, but the panel was rendered by all of these extensions eventually leaded to the probelms like memory leaks, screen freezing or screen flickering, which result in the isaac-sim crashing.

Our test environment (LXC VM):

  • OS: Ubuntu 22.04 TLS
  • CPU: 16 cores
  • RAM: 256 GiB
  • GPU: NVIDIA L40S
  • GPU driver: NVIDIA 535.183.01
  • Docker image: nvcr.io/nvidia/omniverse/isaac-sim:4.2.0
  • Docker Entrypoint: ./isaac-sim.headless.webrtc.sh -v --allow-root

Extensions we had tried to render the panel(s):

  1. Widget: Rotation & screen flickering

    widget
    widget1200×609 246 KB

  2. UiContainer: Memory leak & screen flickering

    uicontainer
    uicontainer980×552 67.4 KB

  3. OmniPBR + PIL: Screen freeze & screen flickering

    omniPBR_PIL
    omniPBR_PIL1006×567 72.7 KB

  4. OmniSurface + PIL: Screen freeze & screen flickering

    OmniSurface_PIL
    OmniSurface_PIL955×544 67.3 KB

The attached files are some isaac-sim logs of above extensions we had tried, thanks.

1_widget_crash.log (1.9 MB)
2_ui_container.log (8.2 MB)
3_omnipbr_PIL.log (4.3 MB)
4_omnisurface_PIL.log (13.2 MB)

Also, we want to know that which extension the Omniverse Development Team would recommend to render the panel(s), so that the isaac-sim can render panel(s) in long running time without screen freezing, crashing or memory leaks.

3

Your driver is not the recommended one of “535.129.03”
Technical Requirements — Omniverse USD Composer

4

I am not sure what kind of dynamic information panel you have built, but it might be making real-time code callbacks that is overwhelming the software. The machine is clearly not out of memory. So if you just delete this info panel, does the system act normally? Can you send us the usd file? Where does this info panel get its real-time info from?

“We had tried some extensions that can make this work, but the panel was rendered by all of these extensions eventually leaded to the problems like memory leaks, screen freezing or screen flickering, which result in the isaac-sim crashing” - What extensions exactly?

I think this is an issue specifically with this info panel or the extensions are you trying to use to make it. We need to isolate why.

5
  1. Yes, but isaac-sim still got memory leaks issue even if the info is deleted from the panel and disable our developed extension.
  2. We did not have usd file, we just work with Mesh or Material.
  3. Our developed extension acquire real-time data from our internal RESTful API service, and update info to the panel in the isaac-sim.

We use the extension “omni.kit.rx.scene_view.utils-106.1.0”.

6
  1. If you are aware of memory leaks then we should report and ticket that directly yes. Can you provide details of that please.
  2. ??? You have to have a usd file. Everything in kit works with usd and usd files. Even if you import a mesh and a material, that is then working in a usd file. You are not ever saving that information into a usd file? Just opening a mesh and material and then closing it without saving?
  3. What happens when you disable your custom extension? Can you provide it to us privately?

I think we need to quantify the memory leaks with “base” Isaac Sim and then with your custom extension and workflow. It sounds like there may be memory leaks for both.