Confusion Around Omniverse Kit Screenshots in Python - and more

[LMTraina99]

Hi!

I have a few questions related to extension development - putting them in block quotes for for the sake of formatting:

First:

I have an extension that generates scenes and want to implement a function to capture a screenshot from a programmatically generated camera (I have the SDFPath).

It seems there are multiple ways of doing this but I’m running into some roadblocks.

I was hoping there would be a way to directly interface with the movie capture tool with omni.kit.window.movie_capture or omni.capture.MovieCapture

Then it also seemed like I could use omni.isaac.dynamic_control, but the package doesn’t show up when I run from code, only from isaac sim. There’s probably a simple workaround but don’t see the isaac extensions in my code instance.

And when I do have all the libraries I need in isaac sim (which unfortunately breaks some of my extension), I’m still confused by the viewport API(s)

it seems there’s an: omni.kit.viewport
and an: omni.kit.viewport_legacy
but I can’t find viewport capture documentation beyond what’s here and even then I had trouble importing what the documentation suggested: Viewport API — Omniverse Kit 104.0.3 documentation

Any help is greatly appreciated as this is certainly way simpler than I’m making it out to be.

Second:

Can you explain what the 2GB update buffer is? I often get errors in console regarding it right before crashes when I’m doing very large scene updates. Is it the amount of geometry data that is allowed to be altered from frame to frame? My scenes are rather large with ~1.5 million total instances of meshes from a library containing ~300 assets (all quite low-LOD - VRAM doesn’t seem to be an issue). My scene geometry info and memory info are shown here:

Third:

Additionally, when I go to update my scene with this large amount of instances, it takes anywhere from 20s-1min. I would like to script my scene to update, then take a screenshot from a predetermined angle (or 2), then update again and repeat until I’ve reached the end of my simulation. Will the asynchronous nature of how kit apps run cause everything to implode when I do this? I’m afraid Omniverse will try taking the screenshot when the scene has not finished updating yet, especially if I’m directly using the viewport API. Welcoming any advice/tips/tricks on how to avoid headaches on automating heavier scenes.

Thank you for your time and I greatly appreciate insight on any of these questions!

-Matthew

[mati-nvidia]

Hi Matthew. Looking into this for you.

[mati-nvidia]

Sorry for the delay on this.

1 & 3) I believe they all use the ViewportAPI.schedule_capture API under the hood. You can see it used directly in omni.kit.viewport.utility.capture_viewport_to_file(). I would take a look at the MovieCapture implementation though to understand the timing for making sure the stage is loaded and sampling is finished.
2) Can you share an example of the output? Are you using instanceable=true for your assets that are being instanced?
3) When you’re make a lot of updates programmatically, the USD update events can become a bottleneck. We’re developing usdrt to give you a faster runtime experience. The API is similar to working with USD, but it updates the Fabric runtime data instead: USD, Fabric, and USDRT — usdrt 7.1.0 documentation