Please find an archive of some scenes and Docker/code/docker compose stuff here: replicator_repro.zip Google Drive file. Using docker compose with NVIDIA Container Toolkit on Linux I have been able to generate motion blurred images using Path Tracing, whi…

Looks like the 65K limit is the issue. here is the fix: render.py import logging import os import sys from pathlib import Path import carb import numpy as np import omni.replicator.core as rep import omni.syntheticdata as sd import omni.usd from pxr import Semantics # Note: Import after kit is …

Getting a lot of 2024-05-23 05:43:25 [560,338ms] [Error] [omni.syntheticdata.plugin] OgnSdStageSemanticInstanceMapping missing filteredLabelMap semLabelMap in the history.

[image] samuel.hodge: tail -f /root/.nvidia-omniverse/logs/Kit/Isaac-Sim/2023.1/*.log Hard crash 2024-05-23 05:47:35 [299,500ms] [Error] [omni.syntheticdata.plugin] OgnSdStageSemanticInstanceMapping missing filteredLabelMap semLabelMap in the history. 2024-05-23 05:47:38 [302,162ms] [Info] […

Posting related report [image] 2024-05-21 17:46:46 [Error] [omni.syntheticdata.plugin] OgnSdStageSemanticInstanceMapping missing filteredLabelMap semLabelMap in the history Isaac Sim Hi, I am getting a flooding of this error in my terminal after modifying the …

@samuel.hodge For the crash, would you be able to post the full log? How reproducible is this? Does it consistently crash at the same time? A small repro would be ideal. For OgnSdStageSemanticInstanceMapping missing filteredLabelMap semLabelMap in the history do you have anything with a semantic la…

The error is consistent on the HPC cluster with driver 535 but not reproducible on my workstation with driver 545. I will supply you with full logs from the three out of twenty scenes on which it occured. you have had my source to create the sematic labels since Monday with the first post. edit: …

[image] samuel.hodge: [6535193.687038] NVRM: Xid (PCI:0000:c1:00): 109, pid=2877153, name=kit, Ch 00000021, errorString CTX SWITCH TIMEOUT, Info 0x108010 The driver issue [6535193.687038] NVRM: Xid (PCI:0000:c1:00): 109, pid=2877153, name=kit, Ch 00000021, errorString CTX SWITCH TIMEOUT, Inf…

hopefully updatiung the kernel driver and the GPU operator to 545+ like the workstation makes it go away. but then again the microk8s on the workstation uses --driver host not a GPU operator like the production HPC.

Sorry I had another look after more coffee and I noticed that the error is conistent between microk8s on my workstation with kernel 545 and on the HPC cluster with 535. neither hardware could complete the work without error on 3//20 randomly rolled scenes it may come down to a particular asset whi…

Driver versions for Linux are /home/sam/Downloads/kit_20240523_203154.log:| NVIDIA-SMI 545.29.06 Driver Version: 545.29.06 CUDA Version: 12.3 | /home/sam/Downloads/kit_20240523_203154.log:| Driver Version: 545.29.06 | Graphics API: Vulkan and /home/sam/Downloads/kit_202405…

Sorry exit code is 137 means not enough memory eg OOM see [image] What Exit Code 137 means for Kubernetes | Refine We'll discuss what exit code 137 means for Kubernetes, how it reflects the operating system's intervention due to memory concerns, and how to mitigate memory issues. …

Docker data generation: Replicator 1.6.3, Pathtraced WITHOUT motion blur? How?

Omniverse Synthetic Data Generation (SDG)

samuel.hodge May 23, 2024, 7:21pm 66

The driver issue

[6535193.687038] NVRM: Xid (PCI:0000:c1:00): 109, pid=2877153, name=kit, Ch 00000021, errorString CTX SWITCH TIMEOUT, Info 0x108010

XID :109 is kinda well documented 33k views

Topic		Replies	Views
RTX Pathtracer crash RTX Renderer	16	1139	April 19, 2024
Pathtraced Renderer Fails in Omniverse Container Synthetic Data Generation (SDG)	4	525	July 18, 2024
Significant performance decrease when using Docker Isaac Sim docker	1	1341	November 28, 2022
Hydra failures on Docker Container Isaac Sim docker	10	1459	October 31, 2022
Isaac Sim Replicator crashes Isaac Sim synthetic-data	10	2947	June 21, 2022
Degradation of rendering capabilities of Isaac-sim on 4.2.0 and 4.1.0 Isaac Sim isaac-sim-v4-2-0 , isaac-sim-v4-1-0	8	673	April 19, 2026
Isaac sim 2020.1 Early Access Docker Container warnings? Isaac Sim	3	742	July 17, 2020
SegFault when launching the docker nvcr.io/nvidia/isaac-sim:2020.2_ea Isaac Sim docker	11	1077	December 5, 2020
DLI Course SDG: RGB files generated by 0_generate_data_headless are completely black Synthetic Data Generation (SDG) dli	1	100	July 31, 2025
Ghosts and other weird things Synthetic Data Generation (SDG)	20	1246	July 4, 2024

Docker data generation: Replicator 1.6.3, Pathtraced WITHOUT motion blur? How?

Related topics