Using RTX Path Tracer as Hydra interface in simple USD application usdrecord

Yeah it doesn’t work

Code for

from omni.kit import app
from omni.kit.window import movie_maker
from omni.kit.file import open_stage
from pathlib import Path
import click
import sys

kaolin_app =

def capture_usd_scene(
    usd_file: str,
    output_file_pattern: str,
    camera: str,
    start_frame: int = 0,
    end_frame: int = 120,
    resolution_width: int = 1920,
    resolution_height: int = 1080,
    Create Frames from USD file through camera for time range
    result = open_stage(Path(usd_file))
    print("Hoopla\n" * 20)

if __name__ == "__main__":
    # if len(sys.argv) > 3:
    #     capture_usd_scene(
    #         usd_file=sys.argv[1],
    #         output_file_pattern=sys.argv[2],
    #         camera=sys.argv[3],
    #     )
    print(f"Num args {len(sys.argv)}\n" * 20)

Called as follows

/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/ --exec "/home/sam/Desktop/ /media/sam/trainer/datasets/simple4.usd wow/frame.####.png 0 120" --no-window

The output is as follows

/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/ --exec "/home/sam/Desktop/ usd_file=/media/sam/trainer/datasets/simple4.usd wow/frame.####.png 0 120" --no-window
[Info] [carb] Logging to file: /home/sam/.nvidia-omniverse/logs/Kit/Kaolin/2021.1/kit_20210805_143528.log
[Info] [carb] Initializing plugin: (interfaces: [carb::events::IEvents v0.2]) (impl:
[0.022s] Loading user config located at: '/media/sam/trainer/omniverse/share/ov/data/Kit/Kaolin/2021.1/user.config.json'
[0.166s] [ext: omni.assets-0.0.0] startup
[0.168s] [ext: omni.kit.window.splash-0.0.0] startup
[0.202s] [ext: omni.kit.async_engine-0.0.0] startup
[0.216s] [ext: omni.kit.agent-0.1.0] startup
[0.218s] [ext: omni.kit.splash-0.1.0] startup
[0.229s] [ext: omni.kit.loop-default-0.1.0] startup
[0.230s] [ext: omni.kit.test-0.0.0] startup
[0.245s] [ext: omni.kit.pipapi-0.0.0] startup
[0.251s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.257s] [ext: omni.stats-0.0.0] startup
[0.261s] [ext: omni.appwindow-0.1.0-rc1] startup
[0.282s] [ext: omni.ui-2.1.2] startup
[0.299s] [ext: omni.kit.commands-0.0.0] startup
[0.306s] [ext: omni.timeline-0.0.0] startup
[0.309s] [ext:] startup
[0.313s] [ext: omni.usd-0.1.8] startup
[0.522s] [ext: omni.client-0.1.0] startup
[0.543s] [ext: omni.kit.tagging-0.1.0] startup
[0.843s] [ext: omni.kit.widget.path_field-2.0.0] startup
[0.851s] [ext: omni.kit.stage_templates-1.0.3] startup
[0.857s] [ext: omni.kit.widget.browser_bar-2.0.0] startup
[0.861s] [ext: omni.kit.window.popup_dialog-1.0.0] startup
[0.869s] [ext: omni.ansel-0.0.0] startup
2021-08-05 05:05:29 [848ms] [Warning] [omni.ext.plugin] [ext: omni.ansel-0.0.0] Failed to load plugins, folder doesn't exist: '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_exts/omni.ansel/bin'.
2021-08-05 05:05:29 [848ms] [Warning] [omni.ext.plugin] [ext: omni.ansel-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_exts/omni.ansel/bin/*.plugin' in '[[native.plugin]]' was not found.
[0.871s] [ext: omni.kit.search_core-1.0.0] startup
[0.875s] [ext: omni.renderer-rtx-0.0.0] startup
[0.877s] [ext: omni.hydra.iray-0.1.0] startup
[0.881s] [ext: omni.hydra.rtx-0.1.0] startup
[0.886s] [ext: omni.kit.widget.filebrowser-2.0.3] startup
[0.895s] [ext: omni.kit.editor-full-0.0.0] startup
2021-08-05 05:05:29 [943ms] [Warning] [omni.ext.plugin] [ext: omni.kit.editor-full-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/rtx.reshade.plugin' in '[[native.plugin]]' was not found.

| Driver Version: 460.67 | Graphics API: Vulkan
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
| 0   | GeForce RTX 3090                 | Yes    |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 2204      | 29266a3c.. |
| 1   | GeForce RTX 3090                 |        |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 2204      | 659eb6f6.. |
| OS: Linux rotobot-ampere-dev, Version: 5.8.0-63-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12009000 (
| Processor: AMD EPYC-Rome Processor | Cores: Unknown | Logical: 48
| Total Memory (MB): 98332 | Free Memory: 74114
| Total Page/Swap (MB): 979 | Free Page/Swap: 979
[4.605s] [ext: omni.kit.window.filepicker-2.0.1] startup
[4.649s] [ext: omni.kit.window.file-1.0.1] startup
[4.710s] [ext:] startup
[4.719s] [ext: omni.kit.material.library-1.0.3] startup
[4.730s] [ext: omni.kit.window.content-1.0.0] startup
[4.740s] [ext: omni.kit.context_menu-1.0.5] startup
[4.745s] [ext: omni.kit.widget.stage-2.3.3] startup
[4.758s] [ext:] startup
[4.763s] [ext: omni.kit.widget.stage_icons-1.0.0] startup
[4.769s] [ext: omni.kit.widget.layers-0.3.1] startup
[4.789s] [ext: omni.kit.window.stage-2.3.2] startup
[4.796s] [ext:] startup
[4.944s] [ext: omni.kit.window.viewport-0.0.0] startup
2021-08-05 05:05:33 [4,935ms] [Warning] [omni.ext.plugin] [ext: omni.kit.window.viewport-0.0.0] '/media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/rtx.reshade.plugin' in '[[native.plugin]]' was not found.
[4.985s] [ext: omni.kit.window.provide_feedback-0.1.0] startup
[4.994s] [ext: kaolin_app.research.utils-0.0.1] startup
[5.141s] [ext: omni.videoencoding-0.0.0] startup
[5.144s] [ext: kaolin_app.research.dataset_visualizer-0.0.3] startup
[5.152s] [ext:] startup
[5.163s] [ext:] startup
[5.172s] [ext: omni.kvdb-0.0.0] startup
[5.178s] [ext: omni.kit.window.script_editor-1.5.0] startup
[5.180s] [ext: omni.usdphysics-1.1.0] startup
[5.183s] [ext: omni.localcache-0.0.0] startup
[5.188s] [ext: omni.debugdraw-0.0.0] startup
[5.195s] [ext:] startup
[5.202s] [ext: omni.convexdecomposition-1.1.0] startup
[5.209s] [ext: omni.syntheticdata-0.0.0] startup
[5.221s] [ext:] startup
[5.249s] [ext: omni.physx-1.1.0-5.1] startup
[5.374s] [ext: syntheticdata.viz-0.3.4] startup
[5.425s] [ext:] startup
[5.474s] [ext: omni.physx.ui-1.1.0-5.1] startup
[5.486s] [ext: omni.kit.window.toolbar-0.0.0] startup
[5.498s] [ext: omni.rtx.window.settings-0.6.0] startup
[5.508s] [ext:] startup
[5.595s] [ext: omni.isaac.dr-0.1.2] startup
[5.620s] [ext: kaolin_app.research.data_generator-0.0.1] startup
[5.641s] [ext: omni.rtx.settings.core-0.5.4] startup
[5.651s] [ext: omni.kit.capture-0.3.2] startup
[5.660s] [ext: omni.kit.filebrowser_column.tags-1.0.0] startup
[5.669s] [ext: omni.kit.window.title-1.0.0] startup
[5.678s] [ext: omni.kit.window.about-0.2.1] startup
[5.686s] [ext: omni.kit.window.movie_maker-0.3.1] startup
[5.697s] [ext: omni.kit.widget.graph-1.0.2] startup
[5.710s] [ext: kaolin_app.research.training_visualizer-0.0.3] startup
[5.720s] [ext: omni.kit.selection-0.0.0] startup
[5.728s] [ext:] startup
[5.731s] [ext: omni.kit.window.status_bar-0.0.0] startup
[5.733s] [ext: omni.kit.window.extensions-1.0.2] startup
[5.747s] [ext: kaolin_app.research.setup-0.0.0] startup
[5.755s] [ext: kaolin-2021.1.0-rc.13] startup
[5.755s] app started
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1
Num args 1

[5.802s] [ext: kaolin-2021.1.0-rc.13] shutdown
[5.802s] [ext: omni.kit.window.movie_maker-0.3.1] shutdown
[5.817s] [ext: omni.kit.capture-0.3.2] shutdown
[5.833s] [ext:] shutdown
[5.849s] [ext: kaolin_app.research.data_generator-0.0.1] shutdown
2021-08-05 05:05:34 [5,837ms] [Error] [carb.python] kaolin_app.research.data_generator-0.0.1 -> <class 'kaolin_app.research.data_generator.extension.Extension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'cell'>, id: 140373424380952", "[2]:type: <class 'method'>, id: 140373424633416", "[3]:type: <class 'method'>, id: 140373691312136"]
[5.874s] [ext:] shutdown
2021-08-05 05:05:34 [5,864ms] [Error] [carb.python] -> <class ''>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 144935448"]
[5.901s] [ext:] shutdown
[5.917s] [ext:] shutdown
[5.933s] [ext:] shutdown
[5.951s] [ext: kaolin_app.research.setup-0.0.0] shutdown
[5.966s] [ext: kaolin_app.research.dataset_visualizer-0.0.3] shutdown
2021-08-05 05:05:34 [5,955ms] [Error] [carb.python] kaolin_app.research.dataset_visualizer-0.0.3 -> <class 'kaolin_app.research.dataset_visualizer.main.KaolinDatasetVisualize'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'method'>, id: 140378993676872", "[2]:type: <class 'cell'>, id: 140373743149272"]
[5.999s] [ext: kaolin_app.research.training_visualizer-0.0.3] shutdown
2021-08-05 05:05:34 [5,991ms] [Error] [carb.python] kaolin_app.research.training_visualizer-0.0.3 -> <class 'kaolin_app.research.training_visualizer.main.KaolinTrainingVisualizer'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'method'>, id: 140373423578504", "[2]:type: <class 'method'>, id: 140373423004168", "[3]:type: <class 'cell'>, id: 140372149478216"]
[6.024s] [ext: kaolin_app.research.utils-0.0.1] shutdown
[6.037s] [ext: omni.kit.widget.layers-0.3.1] shutdown
[6.050s] [ext: omni.kit.window.stage-2.3.2] shutdown
[6.062s] [ext: omni.physx.ui-1.1.0-5.1] shutdown
[6.073s] [ext: omni.physx-1.1.0-5.1] shutdown
[6.086s] [ext: omni.kit.widget.stage-2.3.3] shutdown
[6.097s] [ext: syntheticdata.viz-0.3.4] shutdown
2021-08-05 05:05:34 [6,088ms] [Error] [carb.python] syntheticdata.viz-0.3.4 -> <class 'syntheticdata.viz.viz.SyntheticDataVisualization'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'cell'>, id: 140373692264712"]
[6.121s] [ext: omni.kit.window.viewport-0.0.0] shutdown
2021-08-05 05:05:34 [6,101ms] [Warning] [omni.kit.ui] Failed to remove menu with path: Window/New Viewport Window. It doesn't exist.
[6.135s] [ext: omni.kit.context_menu-1.0.5] shutdown
[6.146s] [ext: omni.kit.window.content-1.0.0] shutdown
[6.157s] [ext:] shutdown
[6.168s] [ext: omni.kit.window.extensions-1.0.2] shutdown
[6.179s] [ext: omni.kit.widget.graph-1.0.2] shutdown
[6.190s] [ext:] shutdown
[6.190s] [ext: omni.kit.window.about-0.2.1] shutdown
[6.200s] [ext: omni.kit.window.file-1.0.1] shutdown
[6.211s] [ext:] shutdown
[6.222s] [ext: omni.kit.window.provide_feedback-0.1.0] shutdown
[6.232s] [ext: omni.kit.window.script_editor-1.5.0] shutdown
[6.232s] [ext: omni.isaac.dr-0.1.2] shutdown
[6.243s] [ext: omni.kit.window.toolbar-0.0.0] shutdown
2021-08-05 05:05:34 [6,233ms] [Error] [carb.python] omni.kit.window.toolbar-0.0.0 -> <class 'omni.kit.window.toolbar.toolbar.Toolbar'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 140391140202840"]
[6.265s] [ext: omni.syntheticdata-0.0.0] shutdown
2021-08-05 05:05:34 [6,245ms] [Warning] [omni.kit.ui] Failed to remove menu with path: Extensions/Synthetic Data Sensors. It doesn't exist.
[6.278s] [ext: omni.kit.editor-full-0.0.0] shutdown
2021-08-05 05:05:34 [6,368ms] [Warning] [rtx.shaderdb.plugin] carb::extras::HandleDatabase<TrueType, HandleType>::~HandleDatabase() [with TrueType = rtx::shaderdb::InternalShaderDesc; HandleType = rtx::shaderdb::_ShaderHandle*]: had 249 outstanding handle(s) at shutdown
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/options.28x32.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/layer_save.28x28.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Texture:'/tmp/carb.o9MkTK/cloud.28x28.png' is not released correctly and cause leak.
2021-08-05 05:05:35 [6,675ms] [Warning] [rtx.resourcemanager.plugin] Release above leaking textures on exit.
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/carb_gfx/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/rtx/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/
2021-08-05 05:05:35 [6,746ms] [Warning] [carb] unloadPlugin: Failed to find a plugin at: /media/sam/trainer/omniverse/share/ov/pkg/kaolin-2021.1.0/_build/kit_release/_build/linux-x86_64/release/plugins/
[6.768s] [ext: omni.ansel-0.0.0] shutdown
[6.779s] [ext: omni.kit.filebrowser_column.tags-1.0.0] shutdown
[6.791s] [ext: omni.kit.tagging-0.1.0] shutdown
[6.801s] [ext: omni.rtx.settings.core-0.5.4] shutdown
[6.813s] [ext: omni.rtx.window.settings-0.6.0] shutdown
[6.823s] [ext: omni.kit.window.filepicker-2.0.1] shutdown
[6.832s] [ext: omni.kit.widget.filebrowser-2.0.3] shutdown
[6.842s] [ext: omni.renderer-rtx-0.0.0] shutdown
[6.842s] [ext: omni.client-0.1.0] shutdown
[6.853s] [ext: omni.debugdraw-0.0.0] shutdown
[6.863s] [ext: omni.kit.material.library-1.0.3] shutdown
2021-08-05 05:05:35 [6,851ms] [Error] [carb.python] omni.kit.material.library-1.0.3 -> <class 'omni.kit.material.library.material_library.MaterialLibraryExtension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 138733048"]
[6.882s] [ext: omni.kit.stage_templates-1.0.3] shutdown
[6.891s] [ext: omni.kit.window.status_bar-0.0.0] shutdown
[6.895s] [ext: omni.kit.window.title-1.0.0] shutdown
[6.904s] [ext: omni.usd-0.1.8] shutdown
[6.922s] [ext:] shutdown
[6.922s] [ext: omni.kit.widget.browser_bar-2.0.0] shutdown
[6.931s] [ext: omni.kit.widget.path_field-2.0.0] shutdown
[6.940s] [ext: omni.kit.selection-0.0.0] shutdown
[6.949s] [ext: omni.kit.window.popup_dialog-1.0.0] shutdown
[6.958s] [ext: omni.ui-2.1.2] shutdown
[6.966s] [ext: omni.appwindow-0.1.0-rc1] shutdown
[6.975s] [ext: omni.kit.splash-0.1.0] shutdown
2021-08-05 05:05:35 [6,963ms] [Error] [carb.python] omni.kit.splash-0.1.0 -> <class 'omni.kit.splash.setup.AppSplashExtension'>: extension object is still alive, something holds a reference on it. Referenced by:
["[0]:type: <class 'frame'>, id: 140372149385288", "[1]:type: <class 'frame'>, id: 140398181431144"]
[6.993s] [ext: omni.kit.window.splash-0.0.0] shutdown
2021-08-05 05:05:35 [6,980ms] [Error] [omni.splash.plugin] Splash screen to be closed wasn't found.
[7.002s] [ext: omni.assets-0.0.0] shutdown
[7.002s] [ext: omni.convexdecomposition-1.1.0] shutdown
[7.010s] [ext: omni.hydra.iray-0.1.0] shutdown
[7.010s] [ext: omni.hydra.rtx-0.1.0] shutdown
[7.011s] [ext: omni.kit.agent-0.1.0] shutdown
[7.019s] [ext: omni.kit.commands-0.0.0] shutdown
[7.027s] [ext:] shutdown
[7.035s] [ext: omni.kit.pip_archive-0.0.0] shutdown
[7.044s] [ext: omni.kit.pipapi-0.0.0] shutdown
[7.048s] [ext:] shutdown
[7.053s] [ext: omni.kit.test-0.0.0] shutdown
[7.057s] [ext: omni.kit.async_engine-0.0.0] shutdown
[7.061s] [ext: omni.kit.loop-default-0.1.0] shutdown
[7.062s] [ext: omni.kit.search_core-1.0.0] shutdown
[7.066s] [ext: omni.kit.widget.stage_icons-1.0.0] shutdown
[7.072s] [ext: omni.localcache-0.0.0] shutdown
[7.076s] [ext: omni.kvdb-0.0.0] shutdown
[7.080s] [ext: omni.stats-0.0.0] shutdown
[7.085s] [ext: omni.timeline-0.0.0] shutdown
[7.089s] [ext: omni.usdphysics-1.1.0] shutdown
[7.089s] [ext: omni.videoencoding-0.0.0] shutdown
Segmentation fault (core dumped)

I am getting a little closer to the solution, could you explain where the docs are for these settings

but by the same token, I am not sure how to “Press the buttons via code”

import omni.ui as ui
import carb.settings
from omni.kit.settings import SettingType
from omni.rtx.window.settings.rtx_settings_stack import RTXSettingsStack
from omni.rtx.window.settings.settings_collection_frame import SettingsCollectionFrame


class AntiAliasingSettingsFrame(SettingsCollectionFrame):
    """ Anti-Aliasing """
    def _on_antialiasing_change(self, *_):

    def _build_ui(self):
        antialiasing_ops = []
        if self._settings.get("/ngx/enabled") is True:
            antialiasing_ops = ["Off", "TAA", "FXAA", "DLSS"]
            antialiasing_ops = ["Off", "TAA", "FXAA"]

        self._add_setting_combo("Algorithm", "/rtx/post/aa/op", antialiasing_ops)

        self._change_cb1 ="/rtx/post/aa/op", self._on_antialiasing_change)

        antialiasingOpIdx = self._settings.get("/rtx/post/aa/op")

        if antialiasingOpIdx == 0:
            """ No AA """

        if antialiasingOpIdx == 1:
            """ TAA """
            self._add_setting(SettingType.FLOAT, "Static scaling", "/rtx/post/scaling/staticRatio", 0.33, 1, 0.01)
            self._add_setting(SettingType.INT, "TAA Samples", "/rtx/post/taa/samples", 1, 16)
            self._add_setting(SettingType.FLOAT, "TAA history scale", "/rtx/post/taa/alpha", 0, 1, 0.1)

        if antialiasingOpIdx == 2:
            """ FXAA """
            self._add_setting(SettingType.FLOAT, "Subpixel Quality", "/rtx/post/fxaa/qualitySubPix", 0.0, 1.0, 0.02)
            self._add_setting(SettingType.FLOAT, "Edge Threshold", "/rtx/post/fxaa/qualityEdgeThreshold", 0.0, 1.0, 0.02)
            self._add_setting(SettingType.FLOAT, "Edge Threshold Min", "/rtx/post/fxaa/qualityEdgeThresholdMin", 0.0, 1.0, 0.02)

        if antialiasingOpIdx == 3:
            """ DLSS """
            dlss_opts = ["Performance", "Balanced", "Quality"]
            self._add_setting_combo("Execution mode", "/rtx/post/dlss/execMode", dlss_opts)
            self._add_setting(SettingType.FLOAT, "Sharpness", "/rtx/post/dlss/sharpness", 0.0, 1.0, 0.5)
            self._add_setting(SettingType.BOOL, "Enable Exposure", "/rtx/post/dlss/enableExposure", True)
            self._change_cb2 ="/rtx/post/dlss/enableExposure", self._on_change)
            if self._settings.get("/rtx/post/dlss/enableExposure"):
                self._add_setting(SettingType.BOOL, "Override Exposure", "/rtx/post/dlss/overrideExposure", False)
                self._add_setting(SettingType.FLOAT, "Override Exposure Value", "/rtx/post/dlss/exposure", 0.00001, 1.0, 0.00001)
    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None

class DirectLightingSettingsFrame(SettingsCollectionFrame):
    """ Direct Lighting """
    def _frame_setting_path(self):
        return "/rtx/directLighting/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.BOOL, "Enable Shadows", "/rtx/shadows/enabled")
        self._add_setting(SettingType.BOOL, "Enable Fractional Opacity", "/rtx/shadows/fractionalCutoutOpacity")
        self._add_setting(SettingType.BOOL, "Show Lights", "/rtx/directLighting/showLights")
        self._add_setting(SettingType.BOOL, "Enable Sampled Direct Lighting", "/rtx/directLighting/sampledLighting/enabled")
        self._add_setting(SettingType.BOOL, "Auto-enable Sampled Lighting Above Light Count Threshold", "/rtx/directLighting/sampledLighting/autoEnable")
        self._add_setting(SettingType.INT, "Auto-enable Sampled Lighting:   Light Count Threshold", "/rtx/directLighting/sampledLighting/autoEnableLightCountThreshold")

        self._change_cb1 ="/rtx/directLighting/sampledLighting/enabled", self._on_change)
        self._change_cb2 ="/rtx/directLighting/sampledLighting/autoEnable", self._on_change)

        if not self._settings.get("/rtx/directLighting/sampledLighting/enabled"):
            with ui.CollapsableFrame("Non-Sampled Direct Lighting", height=0):
                with ui.VStack(height=0, spacing=5, style={"VStack": {"margin_width": 10}}):
                    self._add_setting(SettingType.INT, "Shadow Samples per Pixel", "/rtx/shadows/sampleCount", 1, 16)
                    self._add_setting(SettingType.BOOL, "Lower Resolution Shadows Denoiser", "/rtx/shadows/denoiser/quarterRes")
                    self._add_setting(SettingType.BOOL, "Dome Lighting", "/rtx/directLighting/domeLight/enabled")
                    self._add_setting(SettingType.BOOL, "Dome Lighting in Reflections", "/rtx/directLighting/domeLight/enabledInReflections")
                    self._change_cb3 ="/rtx/directLighting/domeLight/enabled", self._on_change)
            with ui.CollapsableFrame("Sampled Direct Lighting", height=0):
                with ui.VStack(height=0, spacing=5, style={"VStack": {"margin_width": 10}}):
                    sampled_lighting_spp_items = {"1": 1, "2": 2, "4": 4, "8": 8}
                    self._add_setting_combo("Samples per Pixel", "/rtx/directLighting/sampledLighting/samplesPerPixel", sampled_lighting_spp_items)
                    self._add_setting(SettingType.BOOL, "Clamp Sample Count to Light Count", "/rtx/directLighting/sampledLighting/clampSamplesPerPixelToNumberOfLights")
                    self._add_setting_combo("Reflections: Light Samples per Pixel", "/rtx/reflections/sampledLighting/samplesPerPixel", sampled_lighting_spp_items)
                    self._add_setting(SettingType.BOOL, "Reflections: Clamp Sample Count to Light Count", "/rtx/reflections/sampledLighting/clampSamplesPerPixelToNumberOfLights")
                    firefly_filter_types = {"None" : "None", "Median" : "Cross-Bilateral Median", "RCRS" : "Cross-Bilateral RCRS"}
                    self._add_setting_combo("Firefly Filter", "/rtx/lightspeed/ReLAX/fireflySuppressionType", firefly_filter_types)
                    self._add_setting(SettingType.BOOL, "History Clamping", "/rtx/lightspeed/ReLAX/historyClampingEnabled")
                    self._add_setting(SettingType.INT, "Denoiser Iterations", "/rtx/lightspeed/ReLAX/aTrousIterations", 1, 10)

    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None
        self._change_cb3 = None

class ReflectionsSettingsFrame(SettingsCollectionFrame):
    """ Reflections """
    def _frame_setting_path(self):
        return "/rtx/reflections/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.FLOAT, "Max Roughness", "/rtx/reflections/maxRoughness", 0.0, 1.0, 0.02)
        self._add_setting(SettingType.INT, "Max Reflection Bounces", "/rtx/reflections/maxReflectionBounces", 0, 100)

class TranslucencySettingsFrame(SettingsCollectionFrame):
    """ Translucency """
    def _frame_setting_path(self):
        return "/rtx/translucency/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.INT, "Max Refraction Bounces", "/rtx/translucency/maxRefractionBounces", 0, 100)
        self._add_setting( SettingType.FLOAT, "Secondary Bounce Roughness Cutoff", "/rtx/translucency/reflectionCutoff", 0.0, 1.0, 0.005)
        self._add_setting(SettingType.BOOL, "Enable Fractional Cutout Opacity", "/rtx/raytracing/fractionalCutoutOpacity")

class CausticsSettingsFrame(SettingsCollectionFrame):
    """ Caustics """
    def _frame_setting_path(self):
        return "/rtx/caustics/enabled"

    def _build_ui(self):
        self._add_setting(SettingType.INT, "Photon Count Multiplier", "/rtx/raytracing/caustics/photonCountMultiplier", 1, 5000)
        self._add_setting(SettingType.INT, "Photon Max Bounces", "/rtx/raytracing/caustics/photonMaxBounces", 1, 20)
        self._add_setting(SettingType.INT, "Filter Iterations", "/rtx/raytracing/caustics/eawFilteringSteps", 0, 10)

class IndirectDiffuseLightingSettingsFrame(SettingsCollectionFrame):
    """ Indirect Diffuse Lighting """
    def _build_ui(self):
        self._add_setting(SettingType.COLOR3, "Ambient Light Color", "/rtx/sceneDb/ambientLightColor")
        self._add_setting(SettingType.FLOAT, "Ambient Light Intensity", "/rtx/sceneDb/ambientLightIntensity", 0.0, 10.0, 0.1)
        self._add_setting(SettingType.BOOL, "Enable Ambient Occlusion (AO)", "/rtx/ambientOcclusion/enabled")
        self._change_cb1 ="/rtx/ambientOcclusion/enabled", self._on_change)
        if self._settings.get("/rtx/ambientOcclusion/enabled"):
            self._add_setting(SettingType.FLOAT, "AO: Ray Length", "/rtx/ambientOcclusion/rayLength", 0.0, 200.0)
            self._add_setting(SettingType.INT, "AO: Minimum Samples per Pixel", "/rtx/ambientOcclusion/minSamples", 1, 16)
            self._add_setting(SettingType.INT, "AO: Maximum Samples per Pixel", "/rtx/ambientOcclusion/maxSamples", 1, 16)
        self._add_setting(SettingType.BOOL, "Enable Indirect Diffuse GI", "/rtx/indirectDiffuse/enabled")
        self._change_cb2 ="/rtx/indirectDiffuse/enabled", self._on_change)
        if self._settings.get("/rtx/indirectDiffuse/enabled"):
            self._add_setting(SettingType.INT, "Samples per Pixel", "/rtx/indirectDiffuse/fetchSampleCount", 0, 4)
            self._add_setting(SettingType.INT, "Max Bounces", "/rtx/indirectDiffuse/maxBounces", 0, 16)
            self._add_setting(SettingType.FLOAT, "Intensity", "/rtx/indirectDiffuse/scalingFactor", 0.0, 20.0, 0.1)
            self._add_setting(SettingType.INT, "Denoiser: Kernel radius", "/rtx/indirectDiffuse/denoiser/kernelRadius", 1, 64)
            self._add_setting(SettingType.INT, "Denoiser: Iteration count", "/rtx/indirectDiffuse/denoiser/iterations", 1, 10)
            self._add_setting(SettingType.INT, "Denoiser: Max History Length", "/rtx/indirectDiffuse/denoiser/temporal/maxHistory", 1, 100)

    def destroy(self):
        self._change_cb1 = None
        self._change_cb2 = None

class RTMultiGPUSettingsFrame(SettingsCollectionFrame):
    """ Multi-GPU """
    def _frame_setting_path(self):
        return "/rtx/realtime/mgpu/enabled"

    def _build_ui(self):
        currentGpuCount = self._settings.get("/renderer/multiGpu/currentGpuCount")
        self._add_setting(SettingType.INT, "Tile Count", "/rtx/realtime/mgpu/tileCount", 2, currentGpuCount)
        self._add_setting(SettingType.BOOL, "GPU 0 Post Process Only", "/rtx/realtime/mgpu/masterPostProcessOnly")
        self._add_setting(SettingType.INT, "Tile Overlap (Pixels)", "/rtx/realtime/mgpu/tileOverlap", 0, 256, 0.1)
        self._add_setting(SettingType.FLOAT, "Fraction of Overlap Pixels to Blend", "/rtx/realtime/mgpu/tileOverlapBlendFraction", 0.0, 1.0, 0.001)

class RTSettingStack(RTXSettingsStack):
    def __init__(self) -> None:
        self._stack = ui.VStack(spacing=7)
        with self._stack:
            AntiAliasingSettingsFrame("Anti-Aliasing", parent=self)
            DirectLightingSettingsFrame("Direct Lighting", parent=self)
            ReflectionsSettingsFrame("Reflections", parent=self)
            TranslucencySettingsFrame("Translucency", parent=self)
            CausticsSettingsFrame("Caustics", parent=self)
            IndirectDiffuseLightingSettingsFrame("Indirect Diffuse Lighting", parent=self)
            gpuCount = carb.settings.get_settings().get("/renderer/multiGpu/currentGpuCount")
            if gpuCount and gpuCount>1:
                RTMultiGPUSettingsFrame("Multi-GPU", parent=self)

For instance when you use Arnold Renderer you can look at the SDK.

Seems I have found the final entry point

    def _on_capture_sequence_clicked(self):


Now I need to meddle with the GUI code to get it in a fit state to press that button and hopefully all will be well.

It is a pretty large code base with some pretty funky patterns going on, but I found my way to the exit, more tomorrow.

That is the final button press is within reach.

Also I found some other nasty things where the CLI invokation will not shutdown cleanly with Is this a known issue?

The real work is done here


I need some help, how to I emit an event?

I need to send a “click” event to the render to frames button, but I have not been able to find how this is possible.

It seems that the only way to use a omni.kit.window.movie_maker function is through the GUI.

@eoinm I am now stuck.

Hi Sam,
sorry for the delayed reply…
Let me try and respond to the various questions separately.

With the shutdown crash, it may be caused by the way you are shutting down. You can try: instead of what you have and see if that helps,
or alternatively from the command line, you can pass:
--/app/quitAfter=10 which will wait 10 kit updates before quitting

Generally, Kit apps are more complex to work with than a simple synchronous python library… apart from being async, Kit is designed more around running as a server or a long-lived process than a one-shot script executor… We are using it like this in various places, and we need to improve the documentation on that, and how to work with Kit via scripting in general.

One other tip that might help you (and make startup and shutdown easier) is you can start up your Kit app with no extensions, and then enable what you need by calling:
kit --empty --enable omni.kit.window.movie_maker
This will require some experimentation, as there may be some dependencies you need that are not specified explicitly, but worth trying

For what you are trying to do, I agree that going directly to omni.kit.capture is probably the best option, to avoid having to mess around with triggering UI buttons. There is a class there called VideoGenerationHelper that looks like it might do some of what you want.
Most of the render settings are documented in RTX Render Settings — Omniverse RTX documentation.

I think what you want to do is pretty basic, and should be included “out of the box” with Kit apps… I’ve filed an internal ticket to address this soon. In the meantime, when I get a spare hour I can try mocking up something simple to try and save you some time.

Thanks for persisting and good luck

One other tip which might help: render settings (and in fact most settings) can be set directly via the command line.
You can find the name of the setting of interest by hovering over it in the Render Settings widget, e.g

Now, to set this when launching a Kit app, just do:
kit.exe --/rtx/multiThreading/enabled=1


This is good information.

I will continue on seeing what is possible with triggering the GUI in a batch session.

I am familiar enough with asyncio as I have used it in the FastAPI package

So hopefully I can get this thing airbourne.

But I am used to a DCC’s API where you can call functionality as both a batch session (usually first) and then latter make those some calls react to signals emitted from user input on a GUI/Keyboard/mouse event.

But within Kit to date I can only see how much can be done as triggered by a user. Given that I have worked at studios at scale often the user input is encapsulated to a data structure and combined with other data structures at build time to allow you to propagate small changes to thousands of locations on disk without visiting each of those locations via the mouse and keyboard.

Right now I am seeing that this pattern is somewhat prohibitive until I have found the knack of how that can be done.

Tried again

from pathlib import Path
import omni
import carb
import os
import time
import asyncio
import omni.kit.editor
import omni.kit.file
import omni.kit.viewport
import omni.timeline
import logging

def setup_logging(level=logging.INFO):

    urllib3_logger = logging.getLogger("urllib3")


def wait_for_image_writing(frame_path):
    # wait for the last frame is written to disk
    # my tests of scenes of different complexity show
    # a range of 0.2 to 1 seconds wait time
    # so 2 second max time should be enough
    # and we can early quit by checking the last
    # frame every 0.1 seconds.
    seconds_tried = 0.0
    carb.log_info("Waiting for frames to be ready for encoding.")
    while seconds_tried < SECONDS_TO_WAIT:
        if os.path.isfile(frame_path) and os.access(frame_path, os.R_OK):
            seconds_tried += SECONDS_EACH_TIME

async def capture_usd_scene(
    usd_file: str = "./data/raw/shipsShaded.usd",
    output_file_pattern: str = "data/omniverse/capture/frame.####.png",
    camera: str = "/Camera/Camera_001",
    start_frame: int = 1,
    end_frame: int = 10,
    resolution_width: int = 1920,
    resolution_height: int = 1080,
    Create Frames from USD file through camera for time range
    omni.kit.file.open_stage((Path(__file__).parent.parent / Path(usd_file)).as_posix())
    if not Path(usd_file).is_file():
            f"ERROR {Path(usd_file).as_posix()} is not a file, should be a USD file"
    editor = omni.kit.editor.get_editor_interface()
    settings = carb.settings.get_settings()
    timeline = omni.timeline.get_timeline_interface()
    viewport = omni.kit.viewport.get_viewport_interface()
    settings.set_bool("/persistent/app/captureFrame/viewport", True)
    settings.set_int("/app/renderer/resolution/width", resolution_width)
    settings.set_int("/app/renderer/resolution/height", resolution_height)
    settings.set_bool("/app/captureFrame/setAlphaTo1", False)
    settings.set_string("/renderer/active", "rtx")
    settings.set_string("/rtx/rendermode", "RaytracedLighting")
    settings.set_int("/persistent/app/viewport/displayOptions", 0)
    settings.set_bool("/app/asyncRendering", False)
    settings.set_bool("/app/asyncRenderingLowLatency", False)
    settings.set_int("/rtx/pathtracing/spp", 0)
    settings.set_bool("/rtx/resetPtAccumOnlyWhenExternalFrameCounterChanges", True)
    settings.set_int("/rtx/externalFrameCounter", start_frame)
    frame_path = output_file_pattern.split(".")[0]
    Path(frame_path).parent.mkdir(parents=True, exist_ok=True)
    for i in range(start_frame, end_frame, 1):
        settings.set_int("/rtx/externalFrameCounter", i)
        frame_name = f"{frame_path}.{i:04d}.png"
        wait_for_image_writing(frame_name)"Frame {i} output to {frame_name}")"DONE!")

if __name__ == "__main__":
    futures = [capture_usd_scene()]
    loop = asyncio.get_event_loop()

No images are found in the folder, what did I do wrong?

Just trying your code now…

HI Sam,
quick (leading question)… is there any reason you’re using Kaolin rather than Create? Create is built against a more recent version of the Kit SDK, and I noticed running your script that some things might be a bit easier in a newer SDK.

Just Kaolin was working when Create (Beta) was not

I can find the crash bugs and link here

Interesting… Yes if you have any logs etc that would be great.

Hi saleskqr1v,
Several questions regarding your code:

  1. you increase timeline’s current time by 1 per step, unless you want to forward your scene by 1 second per frame, then you may want to give it the actual animation time in seconds, instead of frame number.
  2. have you tried to call the frame capture api only, without any other set up code? Like in Create, there is no editor so you will need omni.renderer_capture:
import omni
import carb
import omni.renderer_capture
import omni.kit.viewport

frame_path = "d:\capture_test.png"
renderer = omni.renderer_capture.acquire_renderer_capture_interface()
viewport_interface = omni.kit.viewport.acquire_viewport_interface()
viewport_rp = viewport_interface.get_viewport_window(None).get_drawable_ldr_resource()
renderer.capture_next_frame_rp_resource(frame_path, viewport_rp)

One way to easily try the image capture code is to run it in the Script Editor window. In Create you can find it with meny [Window - Script Editor]. I don’t have Kaolin installed so I can just guess it’s also there because it a very basic and useful functionality. Also the code to get the right capture interface (editor or render_capture) is in omni.kit.capture extension’s on_startup, for your information.

Also, I can see the you found out that omni.kit.capture is the one that does the actual capture work, this is great. So have you tried to refer to omni.kit.window.movie_maker, see how it uses omni.kit.capture to capture images? The basic workflow is to get the capture extension instance, feed it with capture options and then start capture.

Hope this can help and please let us know what problem you still have.


Sorry I will update later

I did make a solution using capture that I should be able to share

It makes use of the Isaac Omniverse Kit Helper and calling .update() a number of times and waiting for the file to open in an event loop

Most of the problems relate to kit’s async behavior and the event loop


This is what I mixed up

#!/usr/bin/env python

import copy
import os
import omni
import numpy as np
from omni.isaac.python_app import OmniKitHelper
from pathlib import Path
import matplotlib.pyplot as plt
import logging
import asyncio

SEED = 1337

EXAMPLE = Path(__file__).parent.parent / Path("./data/raw/shipsShaded.usda")

def setup_logging(level=logging.INFO):

    urllib3_logger = logging.getLogger("urllib3")


async def load_stage(path):
    await omni.usd.get_context().open_stage_async(path)

def _setup_world(kit, scenario_path):
    # Load scenario
    setup_task = asyncio.ensure_future(load_stage(scenario_path))
    while not setup_task.done():
        #"Update in progress")
    kit.update()"Stage opened with {scenario_path}")


def main(
    output_folder = Path(output_path).parent
    output_folder.mkdir(parents=True, exist_ok=True)"Start up")
    kit = OmniKitHelper(
            "renderer": "PathTracedLighting",
            "experience": f'{os.environ["EXP_PATH"]}/omni.isaac.sim.python.kit',
            "width": width,
            "height": hieght,
    )"Start End")

    from pxr import Semantics  # Note: Import after kit is instantiated
    from omni.syntheticdata import visualize, helpers  # Note: as above
    from omni.isaac.synthetic_utils import (
    )  # Note: as above

    sd_helper = SyntheticDataHelper()"Imports End")
    _setup_world(kit=kit, scenario_path=usd_file)
    stage = kit.get_stage()"Open Stage End")
    for prim in stage.Traverse():
        if prim.GetTypeName() == "Mesh":
        sem = Semantics.SemanticsAPI.Apply(prim, "Semantics")
        prim_type = prim.GetTypeName()
        if str(prim_type) == "Mesh":
            if background_match in str(prim.GetPath()):
      "Background Mesh: {prim.GetPath()}")
      "Foreground Mesh: {prim.GetPath()}")"Traverse Stage End")

    kit.update()"Update End")
    viewport = omni.kit.viewport.get_default_viewport_window()"Viewport End")
    timeline = omni.timeline.get_timeline_interface()
    for i in range(1000):
    for cur_frame in range(start_frame + 1, end_frame + 1, 1):
        for i in range(20):
        gt = sd_helper.get_groundtruth(


        # Setup a figure
        _, axes = plt.subplots(2, 4, figsize=(20, 7))
        axes = axes.flat
        for ax in axes:

        # RGB
        for ax in axes[:-1]:

        # DEPTH
        depth_data = np.clip(gt["depth"], 0, 255)

        # BBOX2D TIGHT
        axes[2].set_title("BBox 2D Tight")
        rgb_data = copy.deepcopy(gt["rgb"])
        axes[2].imshow(visualize.colorize_bboxes(gt["boundingBox2DTight"], rgb_data))

        # BBOX2D LOOSE
        axes[3].set_title("BBox 2D Loose")
        rgb_data = copy.deepcopy(gt["rgb"])
        axes[3].imshow(visualize.colorize_bboxes(gt["boundingBox2DLoose"], rgb_data))

        axes[4].set_title("Instance Segmentation")
        instance_seg = gt["instanceSegmentation"][0]
        instance_rgb = visualize.colorize_segmentation(instance_seg)
        axes[4].imshow(instance_rgb, alpha=0.7)

        axes[5].set_title("Semantic Segmentation")
        semantic_seg = gt["semanticSegmentation"]
        semantic_rgb = visualize.colorize_segmentation(semantic_seg)
        axes[5].imshow(semantic_rgb, alpha=0.7)

        # BBOX 3D
        axes[6].set_title("BBox 3D")
        bbox_3d_data = gt["boundingBox3D"]
        bboxes_3d_corners = bbox_3d_data["corners"]
        projected_corners = helpers.world_to_image(
            bboxes_3d_corners.reshape(-1, 3), viewport
        projected_corners = projected_corners.reshape(-1, 8, 3)
        rgb_data = copy.deepcopy(gt["rgb"])
        bboxes3D_rgb = visualize.colorize_bboxes_3d(projected_corners, rgb_data)

        # Save figure
            + f'/{Path(output_path).name.split(".")[0]}.'
            + f'{cur_frame:04d}.{Path(output_path).name.split(".")[-1]}'
        plt.close()"Frame {cur_frame:04d} Done!")
        # cleanup

if __name__ == "__main__":

Yes if you want to capture images, you may have to disable the kit’s async rendering like this:

        # disable async rendering for capture, otherwise it won't capture images correctly
        self._settings.set_bool("/app/asyncRendering", False)
        self._settings.set_bool("/app/asyncRenderingLowLatency", False)

You can see how its used in omni.kit.capture too.

Thanks for this information it is useful to myself and others