When following the steps for this demo I receive the following errors which cause Omniverse to crash.
https://docs.omniverse.nvidia.com/guide-sdg/latest/case-studies/case1.html
I am using dual 4090 GPUs on Ubuntu 22.04 with a Ryzen 7950 CPU – no official crash dump is created. The extension hangs and then terminates without creating a crash report.
Note, that the Synthetic data is generated successfully, however the extension crashes 10 minutes later:
2024-09-03 20:22:04 [90,428ms] [Warning] [omni.replicator.core.scripts.backends.disk] No root directory specified in carb setting `/omni/replicator/backends/disk/root_dir`. Writing data to /home/blinux/omni.replicator_out/Replicator_Agent
2024-09-03 20:22:04 [90,717ms] [Warning] [omni.hydra] Disabling DLSS Frame Generation for at least one view due to incompatible render outputs and OmniGraph postprocessing being active
2024-09-03 20:22:04 [90,835ms] [Warning] [omni.syntheticdata.plugin] OgnSdPostRenderVarToHost : rendervar copy from texture directly to host buffer is counter-performant. Please use copy from texture to device buffer first.
Module omni.replicator.core.ogn.python._impl.nodes.OgnSemanticSegmentation cc3f83a load on device 'cuda:1' took 0.25 ms
2024-09-03 20:22:11 [98,062ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:22:11 [98,062ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:15 [101,809ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:22:15 [101,809ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:16 [102,771ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:22:16 [102,771ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:37 [123,443ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:22:37 [123,443ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:38 [125,021ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:22:38 [125,021ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:43 [130,176ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:22:43 [130,177ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:54 [141,152ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:22:54 [141,152ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:22:58 [144,303ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:22:58 [144,303ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:23:03 [150,259ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:23:03 [150,259ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:23:18 [164,338ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:23:18 [164,338ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:23:32 [178,366ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:23:32 [178,366ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:23:40 [186,348ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:23:40 [186,348ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:23:42 [188,680ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:23:42 [188,680ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:24:00 [207,230ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:24:00 [207,230ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:24:41 [248,192ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:24:41 [248,192ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:18 [284,587ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:25:18 [284,593ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:27 [293,820ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:25:27 [293,820ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:43 [309,320ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:25:43 [309,320ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:44 [310,527ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:25:44 [310,527ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:53 [319,453ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:25:53 [319,453ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:25:54 [320,509ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:25:54 [320,509ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:26:09 [336,112ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:26:09 [336,112ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:26:16 [342,876ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:26:16 [342,876ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:26:37 [364,298ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:26:37 [364,299ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:26:41 [367,455ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:26:41 [367,455ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:27:24 [410,805ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:27:24 [410,805ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:27:58 [444,327ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:27:58 [444,327ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:28:16 [463,144ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:28:16 [463,144ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:28:43 [490,277ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:28:43 [490,277ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:29:05 [511,933ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:29:05 [511,933ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:29:20 [526,735ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:29:20 [526,735ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:29:29 [535,873ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:29:29 [535,873ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:29:36 [542,668ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:29:36 [542,668ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:29:53 [560,117ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:29:53 [560,117ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:30:02 [569,026ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:30:02 [569,026ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:30:10 [577,143ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:30:10 [577,144ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:30:12 [578,394ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00009
2024-09-03 20:30:12 [578,394ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:30:26 [592,312ms] [Warning] [omni.usd] Hydra engine rtx transitioning from async (/app/runLoops/rendering_0) to synchronous rendering
2024-09-03 20:32:10 [696,313ms] [Error] [carb.graphics-vulkan.plugin] NGX EvaluateFeature failed: 0xbad00005
2024-09-03 20:32:10 [696,313ms] [Error] [carb.graphics-vulkan.plugin] Failed to evaluate DLSS feature.
2024-09-03 20:32:18 [704,311ms] [Error] [carb.cudainterop.plugin] dup failed for resourceType: 3 (file descriptor: 1299)
Too many open files (error: 24)
2024-09-03 20:32:18 [704,311ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2024-09-03 20:32:18 [704,311ms] [Error] [gpu.foundation.plugin] Buffer creation failed for the device: 1.
ExceptionHandler::GenerateDump sys_pipe failed:Too many open files
ExceptionHandler::SendContinueSignalToChild sys_write failed:Bad file descriptor
ExceptionHandler::WaitForContinueSignal sys_read failed:Bad file descriptor
2024-09-03 20:32:18 [0ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred. If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds. This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.