Using RTX Path Tracer as Hydra interface in simple USD application usdrecord

Hi I am trying to get the simplest implementation of a computer vision dataset happening.

I have been able to create a USD scene graph in the Kaolin GUI Application on Unbuntu Linux

Which showed great promise for its ability to do light transport and materials.

Now I would like to manipulate the USD scene graph programatically with Python 3.6 and then serialise the updated scene to disk

From there I would like to output image files to disk

usage: usdrecord [-h] [--mask PRIMPATH[,PRIMPATH...]]
                 [--purposes PURPOSE[,PURPOSE...]] [--camera CAMERA]
                 [--defaultTime | --frames FRAMESPEC[,FRAMESPEC...]]
                 [--complexity {low,medium,high,veryhigh}]
                 [--colorCorrectionMode {disabled,sRGB,openColorIO}]
                 [--renderer {GL}] [--imageWidth IMAGEWIDTH]
                 usdFilePath outputImagePath

Generates images from a USD file

positional arguments:
  usdFilePath           USD file to record
  outputImagePath       Output image path. For frame ranges, the path must
                        contain exactly one frame number placeholder of the
                        form "###" or "###.###". Note that the number of hash
                        marks is variable in each group.

optional arguments:
  -h, --help            show this help message and exit
                        Limit stage population to these prims, their
                        descendants and ancestors. To specify multiple paths,
                        either use commas with no spaces or quote the argument
                        and separate paths by commas and/or spaces.
  --purposes PURPOSE[,PURPOSE...]
                        Specify which UsdGeomImageable purposes should be
                        included in the renders. The "default" purpose is
                        automatically included, so you need specify only the
                        *additional* purposes. If you want more than one extra
                        purpose, either use commas with no spaces or quote the
                        argument and separate purposes by commas and/or
  --camera CAMERA, -cam CAMERA
                        Which camera to use - may be given as either just the
                        camera's prim name (i.e. just the last element in the
                        prim path), or as a full prim path. Note that if only
                        the prim name is used and more than one camera exists
                        with that name, which one is used will effectively be
                        random (default=main_cam)
  --defaultTime, -d     explicitly operate at the Default time code (the
                        default behavior is to operate at the Earliest time
                        specify FrameSpec(s) of the time codes to operate on -
                        A FrameSpec consists of up to three floating point
                        values for the start time code, end time code, and
                        stride of a time code range. A single time code can be
                        specified, or a start and end time code can be
                        specified separated by a colon (:). When a start and
                        end time code are specified, the stride may optionally
                        be specified as well, separating it from the start and
                        end time codes with (x). Multiple FrameSpecs can be
                        combined as a comma-separated list. The following are
                        examples of valid FrameSpecs: 123 - 101:105 - 105:101
                        - 101:109x2 - 101:110x2 - 101:104x0.5
  --complexity {low,medium,high,veryhigh}, -c {low,medium,high,veryhigh}
                        level of refinement to use (default=low)
  --colorCorrectionMode {disabled,sRGB,openColorIO}, -color {disabled,sRGB,openColorIO}
                        the color correction mode to use (default=sRGB)
  --renderer {GL}, -r {GL}
                        Hydra renderer plugin to use when generating images
                        Width of the output image. The height will be computed
                        from this value and the camera's aspect ratio

I would like to be able to specify the --renderer flag to usdrecord to inherit the RTX path tracer renderer from the Kaolin runtime environment.

Alternatively what is the simplest way to make a CLI invokation of a USD scene with materials and lights to a series of images without resorting to clicking buttons in a GUI.

although RTX is a Hydra Render Delegate, it’s not one that you can easily drop into another application that supports Hydra render delegates such as usdview. There are many reasons for this, but the main one is that Omniverse Applications use a slightly custom flavour of USD with some differences in the Hydra scaffolding (e.g to support Vulkan/DirectX etc rather than OpenGL). There’s more about it if you’re interested in Dirk Van Gelder’s talk from Siggraph last year: Open Source at NVIDIA - YouTube.

There is an extension called omni.kit.window.movie_maker that lets you batch capture a series of frames, although it’s a GUI mode extension.

If you wanted to pilot it via command line, I’m sure you could write a fairly simple script along the lines of:
kaolin_exe --no-window --exec " args1 args2 args3"
where the script imports the movie capture extension and calls it directly… Happy to help you out with that - let me know if you get stuck.

Thanks for the nudge in the right direction

Would you suggest staying in Linux or flipping over to windows ?

Hopefully I will have something to update today


Hi Sam,
it probably doesn’t make much difference… I’d stick with Linux if you’re already on that

Yeah it doesn’t work

Code for

from omni.kit import app
from omni.kit.window import movie_maker
from omni.kit.file import open_stage
from pathlib import Path
import click
import sys

kaolin_app =

def capture_usd_scene(
    usd_file: str,
    output_file_pattern: str,
    camera: str,
    start_frame: int = 0,
    end_frame: int = 120,
    resolution_width: int = 1920,
    resolution_height: int = 1080,
    Create Frames from USD file through camera for time range
    result = open_stage(Path(usd_file))
    print("Hoopla\n" * 20)

if __name__ == "__main__":
    # if len(sys.argv) > 3:
    #     capture_usd_scene(
    #         usd_file=sys.argv[1],
    #         output_file_pattern=sys.argv[2],
    #         camera=sys.argv[3],
    #     )
    print(f"Num args {len(sys.argv)}\n" * 20)

Called as follows

/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/ --exec "/home/sam/Desktop/ /media/sam/trainer/datasets/simple4.usd wow/frame.####.png 0 120" --no-window

The output is as follows

/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/ --exec "/home/sam/Desktop/ usd_file=/media/sam/trainer/datasets/simple4.usd wow/frame.####.png 0 120" --no-window
[Info] [carb] Logging to file: /home/sam/.nvidia-omniverse/logs/Kit/Kaolin/2021.1/kit_20210805_143528.log
[Info] [carb] Initializing plugin: (interfaces: [carb::events::IEvents v0.2]) (impl:
[0.022s] Loading user config located at: '/media/sam/trainer/omniverse/share/ov/data/Kit/Kaolin/2021.1/user.config.json'
[0.166s] [ext: omni.assets-0.0.0] startup
[0.168s] [ext: omni.kit.window.splash-0.0.0] startup
[0.202s] [ext: omni.kit.async_engine-0.0.0] startup
[0.216s] [ext: omni.kit.agent-0.1.0] startup
[0.218s] [ext: omni.kit.splash-0.1.0] startup
[0.229s] [ext: omni.kit.loop-default-0.1.0] startup
[0.230s] [ext: omni.kit.test-0.0.0] startup
[0.245s] [ext: omni.kit.pipapi-0.0.0] startup
[0.251s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.257s] [ext: omni.stats-0.0.0] startup
[0.261s] [ext: omni.appwindow-0.1.0-rc1] startup
[0.282s] [ext: omni.ui-2.1.2] startup
[0.299s] [ext: omni.kit.commands-0.0.0] startup
[0.306s] [ext: omni.timeline-0.0.0] startup
[0.309s] [ext:] startup
[0.313s] [ext: omni.usd-0.1.8] startup
[0.522s] [ext: omni.client-0.1.0] startup
[0.543s] [ext: omni.kit.tagging-0.1.0] startup
[0.843s] [ext: omni.kit.widget.path_field-2.0.0] startup
[0.851s] [ext: omni.kit.stage_templates-1.0.3] startup
[0.857s] [ext: omni.kit.widget.browser_bar-2.0.0] startup
[0.861s] [ext: omni.kit.window.popup_dialog-1.0.0] startup
[0.869s] [ext: omni.ansel-0.0.0] startup
2021-08-05 05:05:29 [848ms] [Warning] [omni.ext.plugin] [ext: omni.ansel-0.0.0] Failed to load plugins, folder doesn't exist: '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_exts/omni.ansel/bin'.
2021-08-05 05:05:29 [848ms] [Warning] [omni.ext.plugin] [ext: omni.ansel-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_exts/omni.ansel/bin/*.plugin' in '[[native.plugin]]' was not found.
[0.871s] [ext: omni.kit.search_core-1.0.0] startup
[0.875s] [ext: omni.renderer-rtx-0.0.0] startup
[0.877s] [ext: omni.hydra.iray-0.1.0] startup
[0.881s] [ext: omni.hydra.rtx-0.1.0] startup
[0.886s] [ext: omni.kit.widget.filebrowser-2.0.3] startup
[0.895s] [ext: omni.kit.editor-full-0.0.0] startup
2021-08-05 05:05:29 [943ms] [Warning] [omni.ext.plugin] [ext: omni.kit.editor-full-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/rtx.reshade.plugin' in '[[native.plugin]]' was not found.

| Driver Version: 460.67 | Graphics API: Vulkan
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
| 0   | GeForce RTX 3090                 | Yes    |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 2204      | 29266a3c.. |
| 1   | GeForce RTX 3090                 |        |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 2204      | 659eb6f6.. |
| OS: Linux rotobot-ampere-dev, Version: 5.8.0-63-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12009000 (
| Processor: AMD EPYC-Rome Processor | Cores: Unknown | Logical: 48
| Total Memory (MB): 98332 | Free Memory: 74114
| Total Page/Swap (MB): 979 | Free Page/Swap: 979
[4.605s] [ext: omni.kit.window.filepicker-2.0.1] startup
[4.649s] [ext: omni.kit.window.file-1.0.1] startup
[4.710s] [ext:] startup
[4.719s] [ext: omni.kit.material.library-1.0.3] startup
[4.730s] [ext: omni.kit.window.content-1.0.0] startup
[4.740s] [ext: omni.kit.context_menu-1.0.5] startup
[4.745s] [ext: omni.kit.widget.stage-2.3.3] startup
[4.758s] [ext:] startup
[4.763s] [ext: omni.kit.widget.stage_icons-1.0.0] startup
[4.769s] [ext: omni.kit.widget.layers-0.3.1] startup
[4.789s] [ext: omni.kit.window.stage-2.3.2] startup
[4.796s] [ext:] startup
[4.944s] [ext: omni.kit.window.viewport-0.0.0] startup
2021-08-05 05:05:33 [4,935ms] [Warning] [omni.ext.plugin] [ext: omni.kit.window.viewport-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/rtx.reshade.plugin' in '[[native.plugin]]' was not found.
[4.985s] [ext: omni.kit.window.provide_feedback-0.1.0] startup
[4.994s] [ext: kaolin_app.research.utils-0.0.1] startup
[5.141s] [ext: omni.videoencoding-0.0.0] startup
[5.144s] [ext: kaolin_app.research.dataset_visualizer-0.0.3] startup
[5.152s] [ext:] startup
[5.163s] [ext:] startup
[5.172s] [ext: omni.kvdb-0.0.0] startup
[5.178s] [ext: omni.kit.window.script_editor-1.5.0] startup
[5.180s] [ext: omni.usdphysics-1.1.0] startup
[5.183s] [ext: omni.localcache-0.0.0] startup
[5.188s] [ext: omni.debugdraw-0.0.0] startup
[5.195s] [ext:] startup
[5.202s] [ext: omni.convexdecomposition-1.1.0] startup
[5.209s] [ext: omni.syntheticdata-0.0.0] startup
[5.221s] [ext:] startup
[5.249s] [ext: omni.physx-1.1.0-5.1] startup
[5.374s] [ext: syntheticdata.viz-0.3.4] startup
[5.425s] [ext:] startup
[5.474s] [ext: omni.physx.ui-1.1.0-5.1] startup
[5.486s] [ext: omni.kit.window.toolbar-0.0.0] startup
[5.498s] [ext: omni.rtx.window.settings-0.6.0] startup
[5.508s] [ext:] startup
[5.595s] [ext: omni.isaac.dr-0.1.2] startup
[5.620s] [ext: kaolin_app.research.data_generator-0.0.1] startup
[5.641s] [ext: omni.rtx.settings.core-0.5.4] startup
[5.651s] [ext: omni.kit.capture-0.3.2] startup
[5.660s] [ext: omni.kit.filebrowser_column.tags-1.0.0] startup
[5.669s] [ext: omni.kit.window.title-1.0.0] startup
[5.678s] [ext: omni.kit.window.about-0.2.1] startup
[5.686s] [ext: omni.kit.window.movie_maker-0.3.1] startup
[5.697s] [ext: omni.kit.widget.graph-1.0.2] startup
[5.710s] [ext: kaolin_app.research.training_visualizer-0.0.3] startup
[5.720s] [ext: omni.kit.selection-0.0.0] startup
[5.728s] [ext:] startup
[5.731s] [ext: omni.kit.window.status_bar-0.0.0] startup
[5.733s] [ext: omni.kit.window.extensions-1.0.2] startup
[5.747s] [ext: kaolin_app.research.setup-0.0.0] startup
[5.755s] [ext: kaolin-2021.1.0-rc.13] startup
[5.755s] app started
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1

[5.802s] [ext: kaolin-2021.1.0-rc.13] shutdown
[5.802s] [ext: omni.kit.window.movie_maker-0.3.1] shutdown
[5.817s] [ext: omni.kit.capture-0.3.2] shutdown
[5.833s] [ext:] shutdown
[5.849s] [ext: kaolin_app.research.data_generator-0.0.1] shutdown
2021-08-05 05:05:34 [5,837ms] [Error] [carb.python] kaolin_app.research.data_generator-0.0.1 -> <class 'kaolin_app.research.data_generator.extension.Extension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'cell'>, id: 140373424380952", "[2]:type: <class 'method'>, id: 140373424633416", "[3]:type: <class 'method'>, id: 140373691312136"]
[5.874s] [ext:] shutdown
2021-08-05 05:05:34 [5,864ms] [Error] [carb.python] -> <class ''>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 144935448"]
[5.901s] [ext:] shutdown
[5.917s] [ext:] shutdown
[5.933s] [ext:] shutdown
[5.951s] [ext: kaolin_app.research.setup-0.0.0] shutdown
[5.966s] [ext: kaolin_app.research.dataset_visualizer-0.0.3] shutdown
2021-08-05 05:05:34 [5,955ms] [Error] [carb.python] kaolin_app.research.dataset_visualizer-0.0.3 -> <class 'kaolin_app.research.dataset_visualizer.main.KaolinDatasetVisualize'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'method'>, id: 140378993676872", "[2]:type: <class 'cell'>, id: 140373743149272"]
[5.999s] [ext: kaolin_app.research.training_visualizer-0.0.3] shutdown
2021-08-05 05:05:34 [5,991ms] [Error] [carb.python] kaolin_app.research.training_visualizer-0.0.3 -> <class 'kaolin_app.research.training_visualizer.main.KaolinTrainingVisualizer'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'method'>, id: 140373423578504", "[2]:type: <class 'method'>, id: 140373423004168", "[3]:type: <class 'cell'>, id: 140372149478216"]
[6.024s] [ext: kaolin_app.research.utils-0.0.1] shutdown
[6.037s] [ext: omni.kit.widget.layers-0.3.1] shutdown
[6.050s] [ext: omni.kit.window.stage-2.3.2] shutdown
[6.062s] [ext: omni.physx.ui-1.1.0-5.1] shutdown
[6.073s] [ext: omni.physx-1.1.0-5.1] shutdown
[6.086s] [ext: omni.kit.widget.stage-2.3.3] shutdown
[6.097s] [ext: syntheticdata.viz-0.3.4] shutdown
2021-08-05 05:05:34 [6,088ms] [Error] [carb.python] syntheticdata.viz-0.3.4 -> <class 'syntheticdata.viz.viz.SyntheticDataVisualization'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'cell'>, id: 140373692264712"]
[6.121s] [ext: omni.kit.window.viewport-0.0.0] shutdown
2021-08-05 05:05:34 [6,101ms] [Warning] [omni.kit.ui] Failed to remove menu with path: Window/New Viewport Window. It doesn't exist.
[6.135s] [ext: omni.kit.context_menu-1.0.5] shutdown
[6.146s] [ext: omni.kit.window.content-1.0.0] shutdown
[6.157s] [ext:] shutdown
[6.168s] [ext: omni.kit.window.extensions-1.0.2] shutdown
[6.179s] [ext: omni.kit.widget.graph-1.0.2] shutdown
[6.190s] [ext:] shutdown
[6.190s] [ext: omni.kit.window.about-0.2.1] shutdown
[6.200s] [ext: omni.kit.window.file-1.0.1] shutdown
[6.211s] [ext:] shutdown
[6.222s] [ext: omni.kit.window.provide_feedback-0.1.0] shutdown
[6.232s] [ext: omni.kit.window.script_editor-1.5.0] shutdown
[6.232s] [ext: omni.isaac.dr-0.1.2] shutdown
[6.243s] [ext: omni.kit.window.toolbar-0.0.0] shutdown
2021-08-05 05:05:34 [6,233ms] [Error] [carb.python] omni.kit.window.toolbar-0.0.0 -> <class 'omni.kit.window.toolbar.toolbar.Toolbar'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 140391140202840"]
[6.265s] [ext: omni.syntheticdata-0.0.0] shutdown
2021-08-05 05:05:34 [6,245ms] [Warning] [omni.kit.ui] Failed to remove menu with path: Extensions/Synthetic Data Sensors. It doesn't exist.
[6.278s] [ext: omni.kit.editor-full-0.0.0] shutdown
2021-08-05 05:05:34 [6,368ms] [Warning] [rtx.shaderdb.plugin] carb::extras::HandleDatabase<TrueType, HandleType>::~HandleDatabase() [with TrueType = rtx::shaderdb::InternalShaderDesc; HandleType = rtx::shaderdb::_ShaderHandle*]: had 249 outstanding handle(s) at shutdown
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/options.28x32.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/layer_save.28x28.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/cloud.28x28.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Release above leaking textures on exit.
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/carb_gfx/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/
[6.768s] [ext: omni.ansel-0.0.0] shutdown
[6.779s] [ext: omni.kit.filebrowser_column.tags-1.0.0] shutdown
[6.791s] [ext: omni.kit.tagging-0.1.0] shutdown
[6.801s] [ext: omni.rtx.settings.core-0.5.4] shutdown
[6.813s] [ext: omni.rtx.window.settings-0.6.0] shutdown
[6.823s] [ext: omni.kit.window.filepicker-2.0.1] shutdown
[6.832s] [ext: omni.kit.widget.filebrowser-2.0.3] shutdown
[6.842s] [ext: omni.renderer-rtx-0.0.0] shutdown
[6.842s] [ext: omni.client-0.1.0] shutdown
[6.853s] [ext: omni.debugdraw-0.0.0] shutdown
[6.863s] [ext: omni.kit.material.library-1.0.3] shutdown
2021-08-05 05:05:35 [6,851ms] [Error] [carb.python] omni.kit.material.library-1.0.3 -> <class 'omni.kit.material.library.material_library.MaterialLibraryExtension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 138733048"]
[6.882s] [ext: omni.kit.stage_templates-1.0.3] shutdown
[6.891s] [ext: omni.kit.window.status_bar-0.0.0] shutdown
[6.895s] [ext: omni.kit.window.title-1.0.0] shutdown
[6.904s] [ext: omni.usd-0.1.8] shutdown
[6.922s] [ext:] shutdown
[6.922s] [ext: omni.kit.widget.browser_bar-2.0.0] shutdown
[6.931s] [ext: omni.kit.widget.path_field-2.0.0] shutdown
[6.940s] [ext: omni.kit.selection-0.0.0] shutdown
[6.949s] [ext: omni.kit.window.popup_dialog-1.0.0] shutdown
[6.958s] [ext: omni.ui-2.1.2] shutdown
[6.966s] [ext: omni.appwindow-0.1.0-rc1] shutdown
[6.975s] [ext: omni.kit.splash-0.1.0] shutdown
2021-08-05 05:05:35 [6,963ms] [Error] [carb.python] omni.kit.splash-0.1.0 -> <class 'omni.kit.splash.setup.AppSplashExtension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 140398181431144"]
[6.993s] [ext: omni.kit.window.splash-0.0.0] shutdown
2021-08-05 05:05:35 [6,980ms] [Error] [omni.splash.plugin] Splash screen to be closed wasn't found.
[7.002s] [ext: omni.assets-0.0.0] shutdown
[7.002s] [ext: omni.convexdecomposition-1.1.0] shutdown
[7.010s] [ext: omni.hydra.iray-0.1.0] shutdown
[7.010s] [ext: omni.hydra.rtx-0.1.0] shutdown
[7.011s] [ext: omni.kit.agent-0.1.0] shutdown
[7.019s] [ext: omni.kit.commands-0.0.0] shutdown
[7.027s] [ext:] shutdown
[7.035s] [ext: omni.kit.pip_archive-0.0.0] shutdown
[7.044s] [ext: omni.kit.pipapi-0.0.0] shutdown
[7.048s] [ext:] shutdown
[7.053s] [ext: omni.kit.test-0.0.0] shutdown
[7.057s] [ext: omni.kit.async_engine-0.0.0] shutdown
[7.061s] [ext: omni.kit.loop-default-0.1.0] shutdown
[7.062s] [ext: omni.kit.search_core-1.0.0] shutdown
[7.066s] [ext: omni.kit.widget.stage_icons-1.0.0] shutdown
[7.072s] [ext: omni.localcache-0.0.0] shutdown
[7.076s] [ext: omni.kvdb-0.0.0] shutdown
[7.080s] [ext: omni.stats-0.0.0] shutdown
[7.085s] [ext: omni.timeline-0.0.0] shutdown
[7.089s] [ext: omni.usdphysics-1.1.0] shutdown
[7.089s] [ext: omni.videoencoding-0.0.0] shutdown
Segmentation fault (core dumped)

I am getting a little closer to the solution, could you explain where the docs are for these settings

but by the same token, I am not sure how to “Press the buttons via code”

import omni.ui as ui
import carb.settings
from omni.kit.settings import SettingType
from omni.rtx.window.settings.rtx_settings_stack import RTXSettingsStack
from omni.rtx.window.settings.settings_collection_frame import SettingsCollectionFrame


class AntiAliasingSettingsFrame(SettingsCollectionFrame):
    """ Anti-Aliasing """
    def _on_antialiasing_change(self, *_):

    def _build_ui(self):
        antialiasing_ops = []
        if self._settings.get("/ngx/enabled") is True:
            antialiasing_ops = ["Off", "TAA", "FXAA", "DLSS"]
            antialiasing_ops = ["Off", "TAA", "FXAA"]

        self._add_setting_combo("Algorithm", "/rtx/post/aa/op", antialiasing_ops)

        self._change_cb1 ="/rtx/post/aa/op", self._on_antialiasing_change)

        antialiasingOpIdx = self._settings.get("/rtx/post/aa/op")

        if antialiasingOpIdx == 0:
            """ No AA """

        if antialiasingOpIdx == 1:
            """ TAA """
            self._add_setting(SettingType.FLOAT, "Static scaling", "/rtx/post/scaling/staticRatio", 0.33, 1, 0.01)
            self._add_setting(SettingType.INT, "TAA Samples", "/rtx/post/taa/samples", 1, 16)
            self._add_setting(SettingType.FLOAT, "TAA history scale", "/rtx/post/taa/alpha", 0, 1, 0.1)

        if antialiasingOpIdx == 2:
            """ FXAA """
            self._add_setting(SettingType.FLOAT, "Subpixel Quality", "/rtx/post/fxaa/qualitySubPix", 0.0, 1.0, 0.02)
            self._add_setting(SettingType.FLOAT, "Edge Threshold", "/rtx/post/fxaa/qualityEdgeThreshold", 0.0, 1.0, 0.02)
            self._add_setting(SettingType.FLOAT, "Edge Threshold Min", "/rtx/post/fxaa/qualityEdgeThresholdMin", 0.0, 1.0, 0.02)

        if antialiasingOpIdx == 3:
            """ DLSS """
            dlss_opts = ["Performance", "Balanced", "Quality"]
            self._add_setting_combo("Execution mode", "/rtx/post/dlss/execMode", dlss_opts)
            self._add_setting(SettingType.FLOAT, "Sharpness", "/rtx/post/dlss/sharpness", 0.0, 1.0, 0.5)
            self._add_setting(SettingType.BOOL, "Enable Exposure", "/rtx/post/dlss/enableExposure", True)
            self._change_cb2 ="/rtx/post/dlss/enableExposure", self._on_change)
            if self._settings.get("/rtx/post/dlss/enableExposure"):
                self._add_setting(SettingType.BOOL, "Override Exposure", "/rtx/post/dlss/overrideExposure", False)
                self._add_setting(SettingType.FLOAT, "Override Exposure Value", "/rtx/post/dlss/exposure", 0.00001, 1.0, 0.00001)
    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None

class DirectLightingSettingsFrame(SettingsCollectionFrame):
    """ Direct Lighting """
    def _frame_setting_path(self):
        return "/rtx/directLighting/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.BOOL, "Enable Shadows", "/rtx/shadows/enabled")
        self._add_setting(SettingType.BOOL, "Enable Fractional Opacity", "/rtx/shadows/fractionalCutoutOpacity")
        self._add_setting(SettingType.BOOL, "Show Lights", "/rtx/directLighting/showLights")
        self._add_setting(SettingType.BOOL, "Enable Sampled Direct Lighting", "/rtx/directLighting/sampledLighting/enabled")
        self._add_setting(SettingType.BOOL, "Auto-enable Sampled Lighting Above Light Count Threshold", "/rtx/directLighting/sampledLighting/autoEnable")
        self._add_setting(SettingType.INT, "Auto-enable Sampled Lighting:   Light Count Threshold", "/rtx/directLighting/sampledLighting/autoEnableLightCountThreshold")

        self._change_cb1 ="/rtx/directLighting/sampledLighting/enabled", self._on_change)
        self._change_cb2 ="/rtx/directLighting/sampledLighting/autoEnable", self._on_change)

        if not self._settings.get("/rtx/directLighting/sampledLighting/enabled"):
            with ui.CollapsableFrame("Non-Sampled Direct Lighting", height=0):
                with ui.VStack(height=0, spacing=5, style={"VStack": {"margin_width": 10}}):
                    self._add_setting(SettingType.INT, "Shadow Samples per Pixel", "/rtx/shadows/sampleCount", 1, 16)
                    self._add_setting(SettingType.BOOL, "Lower Resolution Shadows Denoiser", "/rtx/shadows/denoiser/quarterRes")
                    self._add_setting(SettingType.BOOL, "Dome Lighting", "/rtx/directLighting/domeLight/enabled")
                    self._add_setting(SettingType.BOOL, "Dome Lighting in Reflections", "/rtx/directLighting/domeLight/enabledInReflections")
                    self._change_cb3 ="/rtx/directLighting/domeLight/enabled", self._on_change)
            with ui.CollapsableFrame("Sampled Direct Lighting", height=0):
                with ui.VStack(height=0, spacing=5, style={"VStack": {"margin_width": 10}}):
                    sampled_lighting_spp_items = {"1": 1, "2": 2, "4": 4, "8": 8}
                    self._add_setting_combo("Samples per Pixel", "/rtx/directLighting/sampledLighting/samplesPerPixel", sampled_lighting_spp_items)
                    self._add_setting(SettingType.BOOL, "Clamp Sample Count to Light Count", "/rtx/directLighting/sampledLighting/clampSamplesPerPixelToNumberOfLights")
                    self._add_setting_combo("Reflections: Light Samples per Pixel", "/rtx/reflections/sampledLighting/samplesPerPixel", sampled_lighting_spp_items)
                    self._add_setting(SettingType.BOOL, "Reflections: Clamp Sample Count to Light Count", "/rtx/reflections/sampledLighting/clampSamplesPerPixelToNumberOfLights")
                    firefly_filter_types = {"None" : "None", "Median" : "Cross-Bilateral Median", "RCRS" : "Cross-Bilateral RCRS"}
                    self._add_setting_combo("Firefly Filter", "/rtx/lightspeed/ReLAX/fireflySuppressionType", firefly_filter_types)
                    self._add_setting(SettingType.BOOL, "History Clamping", "/rtx/lightspeed/ReLAX/historyClampingEnabled")
                    self._add_setting(SettingType.INT, "Denoiser Iterations", "/rtx/lightspeed/ReLAX/aTrousIterations", 1, 10)

    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None
        self._change_cb3 = None

class ReflectionsSettingsFrame(SettingsCollectionFrame):
    """ Reflections """
    def _frame_setting_path(self):
        return "/rtx/reflections/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.FLOAT, "Max Roughness", "/rtx/reflections/maxRoughness", 0.0, 1.0, 0.02)
        self._add_setting(SettingType.INT, "Max Reflection Bounces", "/rtx/reflections/maxReflectionBounces", 0, 100)

class TranslucencySettingsFrame(SettingsCollectionFrame):
    """ Translucency """
    def _frame_setting_path(self):
        return "/rtx/translucency/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.INT, "Max Refraction Bounces", "/rtx/translucency/maxRefractionBounces", 0, 100)
        self._add_setting( SettingType.FLOAT, "Secondary Bounce Roughness Cutoff", "/rtx/translucency/reflectionCutoff", 0.0, 1.0, 0.005)
        self._add_setting(SettingType.BOOL, "Enable Fractional Cutout Opacity", "/rtx/raytracing/fractionalCutoutOpacity")

class CausticsSettingsFrame(SettingsCollectionFrame):
    """ Caustics """
    def _frame_setting_path(self):
        return "/rtx/caustics/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.INT, "Photon Count Multiplier", "/rtx/raytracing/caustics/photonCountMultiplier", 1, 5000)
        self._add_setting(SettingType.INT, "Photon Max Bounces", "/rtx/raytracing/caustics/photonMaxBounces", 1, 20)
        self._add_setting(SettingType.INT, "Filter Iterations", "/rtx/raytracing/caustics/eawFilteringSteps", 0, 10)

class IndirectDiffuseLightingSettingsFrame(SettingsCollectionFrame):
    """ Indirect Diffuse Lighting """
    def _build_ui(self):
        self._add_setting(SettingType.COLOR3, "Ambient Light Color", "/rtx/sceneDb/ambientLightColor")
        self._add_setting(SettingType.FLOAT, "Ambient Light Intensity", "/rtx/sceneDb/ambientLightIntensity", 0.0, 10.0, 0.1)
        self._add_setting(SettingType.BOOL, "Enable Ambient Occlusion (AO)", "/rtx/ambientOcclusion/enabled")
        self._change_cb1 ="/rtx/ambientOcclusion/enabled", self._on_change)
        if self._settings.get("/rtx/ambientOcclusion/enabled"):
            self._add_setting(SettingType.FLOAT, "AO: Ray Length", "/rtx/ambientOcclusion/rayLength", 0.0, 200.0)
            self._add_setting(SettingType.INT, "AO: Minimum Samples per Pixel", "/rtx/ambientOcclusion/minSamples", 1, 16)
            self._add_setting(SettingType.INT, "AO: Maximum Samples per Pixel", "/rtx/ambientOcclusion/maxSamples", 1, 16)
        self._add_setting(SettingType.BOOL, "Enable Indirect Diffuse GI", "/rtx/indirectDiffuse/enabled")
        self._change_cb2 ="/rtx/indirectDiffuse/enabled", self._on_change)
        if self._settings.get("/rtx/indirectDiffuse/enabled"):
            self._add_setting(SettingType.INT, "Samples per Pixel", "/rtx/indirectDiffuse/fetchSampleCount", 0, 4)
            self._add_setting(SettingType.INT, "Max Bounces", "/rtx/indirectDiffuse/maxBounces", 0, 16)
            self._add_setting(SettingType.FLOAT, "Intensity", "/rtx/indirectDiffuse/scalingFactor", 0.0, 20.0, 0.1)
            self._add_setting(SettingType.INT, "Denoiser: Kernel radius", "/rtx/indirectDiffuse/denoiser/kernelRadius", 1, 64)
            self._add_setting(SettingType.INT, "Denoiser: Iteration count", "/rtx/indirectDiffuse/denoiser/iterations", 1, 10)
            self._add_setting(SettingType.INT, "Denoiser: Max History Length", "/rtx/indirectDiffuse/denoiser/temporal/maxHistory", 1, 100)

    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None

class RTMultiGPUSettingsFrame(SettingsCollectionFrame):
    """ Multi-GPU """
    def _frame_setting_path(self):
        return "/rtx/realtime/mgpu/enabled"

    def _build_ui(self):
        currentGpuCount = self._settings.get("/renderer/multiGpu/currentGpuCount")
        self._add_setting(SettingType.INT, "Tile Count", "/rtx/realtime/mgpu/tileCount", 2, currentGpuCount)
        self._add_setting(SettingType.BOOL, "GPU 0 Post Process Only", "/rtx/realtime/mgpu/masterPostProcessOnly")
        self._add_setting(SettingType.INT, "Tile Overlap (Pixels)", "/rtx/realtime/mgpu/tileOverlap", 0, 256, 0.1)
        self._add_setting(SettingType.FLOAT, "Fraction of Overlap Pixels to Blend", "/rtx/realtime/mgpu/tileOverlapBlendFraction", 0.0, 1.0, 0.001)

class RTSettingStack(RTXSettingsStack):
    def __init__(self) -> None:
        self._stack = ui.VStack(spacing=7)
        with self._stack:
            AntiAliasingSettingsFrame("Anti-Aliasing", parent=self)
            DirectLightingSettingsFrame("Direct Lighting", parent=self)
            ReflectionsSettingsFrame("Reflections", parent=self)
            TranslucencySettingsFrame("Translucency", parent=self)
            CausticsSettingsFrame("Caustics", parent=self)
            IndirectDiffuseLightingSettingsFrame("Indirect Diffuse Lighting", parent=self)
            gpuCount = carb.settings.get_settings().get("/renderer/multiGpu/currentGpuCount")
            if gpuCount and gpuCount>1:
                RTMultiGPUSettingsFrame("Multi-GPU", parent=self)

For instance when you use Arnold Renderer you can look at the SDK.

Seems I have found the final entry point

    def _on_capture_sequence_clicked(self):


Now I need to meddle with the GUI code to get it in a fit state to press that button and hopefully all will be well.

It is a pretty large code base with some pretty funky patterns going on, but I found my way to the exit, more tomorrow.

That is the final button press is within reach.

Also I found some other nasty things where the CLI invokation will not shutdown cleanly with Is this a known issue?

The real work is done here


I need some help, how to I emit an event?

I need to send a “click” event to the render to frames button, but I have not been able to find how this is possible.

It seems that the only way to use a omni.kit.window.movie_maker function is through the GUI.

@eoinm I am now stuck.

Hi Sam,
sorry for the delayed reply…
Let me try and respond to the various questions separately.

With the shutdown crash, it may be caused by the way you are shutting down. You can try: instead of what you have and see if that helps,
or alternatively from the command line, you can pass:
--/app/quitAfter=10 which will wait 10 kit updates before quitting

Generally, Kit apps are more complex to work with than a simple synchronous python library… apart from being async, Kit is designed more around running as a server or a long-lived process than a one-shot script executor… We are using it like this in various places, and we need to improve the documentation on that, and how to work with Kit via scripting in general.

One other tip that might help you (and make startup and shutdown easier) is you can start up your Kit app with no extensions, and then enable what you need by calling:
kit --empty --enable omni.kit.window.movie_maker
This will require some experimentation, as there may be some dependencies you need that are not specified explicitly, but worth trying

For what you are trying to do, I agree that going directly to omni.kit.capture is probably the best option, to avoid having to mess around with triggering UI buttons. There is a class there called VideoGenerationHelper that looks like it might do some of what you want.
Most of the render settings are documented in RTX Render Settings — Omniverse RTX documentation.

I think what you want to do is pretty basic, and should be included “out of the box” with Kit apps… I’ve filed an internal ticket to address this soon. In the meantime, when I get a spare hour I can try mocking up something simple to try and save you some time.

Thanks for persisting and good luck

One other tip which might help: render settings (and in fact most settings) can be set directly via the command line.
You can find the name of the setting of interest by hovering over it in the Render Settings widget, e.g

Now, to set this when launching a Kit app, just do:
kit.exe --/rtx/multiThreading/enabled=1


This is good information.

I will continue on seeing what is possible with triggering the GUI in a batch session.

I am familiar enough with asyncio as I have used it in the FastAPI package

So hopefully I can get this thing airbourne.

But I am used to a DCC’s API where you can call functionality as both a batch session (usually first) and then latter make those some calls react to signals emitted from user input on a GUI/Keyboard/mouse event.

But within Kit to date I can only see how much can be done as triggered by a user. Given that I have worked at studios at scale often the user input is encapsulated to a data structure and combined with other data structures at build time to allow you to propagate small changes to thousands of locations on disk without visiting each of those locations via the mouse and keyboard.

Right now I am seeing that this pattern is somewhat prohibitive until I have found the knack of how that can be done.

Tried again

from pathlib import Path
import omni
import carb
import os
import time
import asyncio
import omni.kit.editor
import omni.kit.file
import omni.kit.viewport
import omni.timeline
import logging

def setup_logging(level=logging.INFO):

    urllib3_logger = logging.getLogger("urllib3")


def wait_for_image_writing(frame_path):
    # wait for the last frame is written to disk
    # my tests of scenes of different complexity show
    # a range of 0.2 to 1 seconds wait time
    # so 2 second max time should be enough
    # and we can early quit by checking the last
    # frame every 0.1 seconds.
    seconds_tried = 0.0
    carb.log_info("Waiting for frames to be ready for encoding.")
    while seconds_tried < SECONDS_TO_WAIT:
        if os.path.isfile(frame_path) and os.access(frame_path, os.R_OK):
            seconds_tried += SECONDS_EACH_TIME

async def capture_usd_scene(
    usd_file: str = "./data/raw/shipsShaded.usd",
    output_file_pattern: str = "data/omniverse/capture/frame.####.png",
    camera: str = "/Camera/Camera_001",
    start_frame: int = 1,
    end_frame: int = 10,
    resolution_width: int = 1920,
    resolution_height: int = 1080,
    Create Frames from USD file through camera for time range
    omni.kit.file.open_stage((Path(__file__).parent.parent / Path(usd_file)).as_posix())
    if not Path(usd_file).is_file():
            f"ERROR {Path(usd_file).as_posix()} is not a file, should be a USD file"
    editor = omni.kit.editor.get_editor_interface()
    settings = carb.settings.get_settings()
    timeline = omni.timeline.get_timeline_interface()
    viewport = omni.kit.viewport.get_viewport_interface()
    settings.set_bool("/persistent/app/captureFrame/viewport", True)
    settings.set_int("/app/renderer/resolution/width", resolution_width)
    settings.set_int("/app/renderer/resolution/height", resolution_height)
    settings.set_bool("/app/captureFrame/setAlphaTo1", False)
    settings.set_string("/renderer/active", "rtx")
    settings.set_string("/rtx/rendermode", "RaytracedLighting")
    settings.set_int("/persistent/app/viewport/displayOptions", 0)
    settings.set_bool("/app/asyncRendering", False)
    settings.set_bool("/app/asyncRenderingLowLatency", False)
    settings.set_int("/rtx/pathtracing/spp", 0)
    settings.set_bool("/rtx/resetPtAccumOnlyWhenExternalFrameCounterChanges", True)
    settings.set_int("/rtx/externalFrameCounter", start_frame)
    frame_path = output_file_pattern.split(".")[0]
    Path(frame_path).parent.mkdir(parents=True, exist_ok=True)
    for i in range(start_frame, end_frame, 1):
        settings.set_int("/rtx/externalFrameCounter", i)
        frame_name = f"{frame_path}.{i:04d}.png"
        wait_for_image_writing(frame_name)"Frame {i} output to {frame_name}")"DONE!")

if __name__ == "__main__":
    futures = [capture_usd_scene()]
    loop = asyncio.get_event_loop()

No images are found in the folder, what did I do wrong?

Just trying your code now…

HI Sam,
quick (leading question)… is there any reason you’re using Kaolin rather than Create? Create is built against a more recent version of the Kit SDK, and I noticed running your script that some things might be a bit easier in a newer SDK.

Just Kaolin was working when Create (Beta) was not

I can find the crash bugs and link here

Interesting… Yes if you have any logs etc that would be great.

Hi saleskqr1v,
Several questions regarding your code:

  1. you increase timeline’s current time by 1 per step, unless you want to forward your scene by 1 second per frame, then you may want to give it the actual animation time in seconds, instead of frame number.
  2. have you tried to call the frame capture api only, without any other set up code? Like in Create, there is no editor so you will need omni.renderer_capture:
import omni
import carb
import omni.renderer_capture
import omni.kit.viewport

frame_path = "d:\capture_test.png"
renderer = omni.renderer_capture.acquire_renderer_capture_interface()
viewport_interface = omni.kit.viewport.acquire_viewport_interface()
viewport_rp = viewport_interface.get_viewport_window(None).get_drawable_ldr_resource()
renderer.capture_next_frame_rp_resource(frame_path, viewport_rp)

One way to easily try the image capture code is to run it in the Script Editor window. In Create you can find it with meny [Window - Script Editor]. I don’t have Kaolin installed so I can just guess it’s also there because it a very basic and useful functionality. Also the code to get the right capture interface (editor or render_capture) is in omni.kit.capture extension’s on_startup, for your information.

Also, I can see the you found out that omni.kit.capture is the one that does the actual capture work, this is great. So have you tried to refer to omni.kit.window.movie_maker, see how it uses omni.kit.capture to capture images? The basic workflow is to get the capture extension instance, feed it with capture options and then start capture.

Hope this can help and please let us know what problem you still have.