Maximum loading time reached while waiting for assets to load. Asset loading may not have completed

Please tell me how to operate the /exts/omni.replicator.core/maxAssetLoadingTime as its undocumented.

This could be a 4.2 regression though, assets seem to take ages to load. Will verify.

cc @jlafleche @hclever @pcallender

Starting Data Generation in /workspaces/Surveily.Datasets.Generator.Omniverse
/isaac-sim
Starting kit application with the following args:  ['/isaac-sim/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/isaac-sim/apps/omni.isaac.sim.python.kit', '--/app/tokens/exe-path=/isaac-sim/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1280', '--/app/renderer/resolution/height=720', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/isaac-sim/exts', '--ext-folder', '/isaac-sim/apps', '--/physics/cudaDevice=0', '--portable', '--no-window', '--/app/window/hideUi=1', '--allow-root']
Passing the following args to the base kit application:  ['--headless', 'True', '--height', '720', '--width', '1280', '--num_frames', '10', '--scene', 'warehouse_small', '--rt_frames', '5']
[Info] [carb] Logging to file: /isaac-sim/kit/logs/Kit/Isaac-Sim/4.2/kit_20240929_180347.log
2024-09-29 18:03:47 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[0.171s] [ext: omni.blobkey-1.1.2] startup
[0.171s] [ext: omni.kit.async_engine-0.0.1] startup
[1.918s] [ext: omni.stats-1.0.1] startup
[1.920s] [ext: omni.datastore-0.0.0] startup
[1.926s] [ext: omni.client-1.2.0] startup
[1.951s] [ext: omni.ujitso.default-1.0.0] startup
[1.952s] [ext: omni.gpu_foundation.shadercache.vulkan-1.0.0] startup
[1.958s] [ext: omni.hsscclient-1.1.1] startup
[1.959s] [ext: omni.assets.plugins-0.0.0] startup
[1.961s] [ext: omni.gpu_foundation-0.0.0] startup
[1.972s] [ext: carb.windowing.plugins-1.0.0] startup
[2.006s] [ext: omni.kit.renderer.init-0.0.0] startup

|---------------------------------------------------------------------------------------------|
| Driver Version: 550.107.02    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 3090          | Yes: 0 |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 2204      | 3272afb7.. |
|     |                                  |        |     |            | 2b        |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 6.8.0-45-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101011 (1.21.1.11)
| Processor: AMD Ryzen 5 5600X 6-Core Processor | Cores: 6 | Logical: 12
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 15876 | Free Memory: 10699
| Total Page/Swap (MB): 4095 | Free Page/Swap: 2299
|---------------------------------------------------------------------------------------------|
[2.661s] [ext: omni.kit.pipapi-0.0.0] startup
[2.664s] [ext: omni.kit.pip_archive-0.0.0] startup
[2.664s] [ext: omni.pip.compute-1.4.0] startup
[2.665s] [ext: omni.pip.cloud-1.1.3] startup
[2.668s] [ext: omni.isaac.core_archive-2.3.0] startup
[2.669s] [ext: omni.isaac.ml_archive-2.0.1] startup
[2.669s] [ext: omni.materialx.libs-1.0.4] startup
[2.677s] [ext: omni.kit.telemetry-0.5.0] startup
[2.709s] [ext: omni.kit.loop-isaac-1.2.0] startup
[2.711s] [ext: omni.kit.test-1.1.0] startup
[2.790s] [ext: omni.usd.config-1.0.4] startup
[2.798s] [ext: omni.gpucompute.plugins-0.0.0] startup
[2.799s] [ext: omni.usd.libs-1.0.1] startup
[2.873s] [ext: omni.iray.libs-0.0.0] startup
[2.885s] [ext: omni.mdl.neuraylib-0.2.8] startup
[2.888s] [ext: omni.mdl-55.0.1] startup
[2.914s] [ext: omni.kit.usd.mdl-1.0.1] startup
[3.026s] [ext: omni.appwindow-1.1.8] startup
[3.027s] [ext: omni.kit.renderer.core-1.0.2] startup
[3.040s] [ext: omni.kit.renderer.capture-0.0.0] startup
[3.043s] [ext: omni.kit.renderer.imgui-1.0.1] startup
[3.120s] [ext: omni.ui-2.25.22] startup
[3.131s] [ext: omni.kit.mainwindow-1.0.3] startup
[3.132s] [ext: carb.audio-0.1.0] startup
[3.136s] [ext: omni.uiaudio-1.0.0] startup
[3.137s] [ext: omni.kit.uiapp-0.0.0] startup
[3.137s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[3.146s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[3.149s] [ext: omni.usd.schema.isaac-2.1.0] startup
[3.156s] [ext: omni.anim.behavior.schema-106.1.0] startup
[3.161s] [ext: omni.usd.schema.anim-0.0.0] startup
[3.187s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[3.189s] [ext: omni.usd.schema.semantics-0.0.0] startup
[3.192s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[3.197s] [ext: omni.usd.schema.physx-106.1.9] startup
[3.226s] [ext: omni.usd.schema.forcefield-106.1.9] startup
[3.231s] [ext: omni.anim.graph.schema-106.1.0] startup
[3.238s] [ext: omni.usd.schema.audio-0.0.0] startup
[3.242s] [ext: omni.anim.navigation.schema-106.1.0] startup
[3.247s] [ext: omni.graph.exec-0.9.4] startup
[3.248s] [ext: omni.kit.actions.core-1.0.0] startup
[3.250s] [ext: omni.kit.exec.core-0.13.4] startup
[3.253s] [ext: omni.kit.commands-1.4.9] startup
[3.258s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[3.260s] [ext: omni.resourcemonitor-105.0.1] startup
[3.262s] [ext: omni.kit.window.popup_dialog-2.0.24] startup
[3.267s] [ext: omni.activity.core-1.0.1] startup
[3.269s] [ext: omni.kit.widget.nucleus_connector-1.1.9] startup
[3.271s] [ext: omni.timeline-1.0.10] startup
[3.273s] [ext: omni.usd_resolver-1.0.0] startup
[3.279s] [ext: usdrt.scenegraph-7.5.0] startup
[3.336s] [ext: omni.usd.core-1.4.1] startup
[3.339s] [ext: omni.kit.usd_undo-0.1.8] startup
[3.340s] [ext: omni.hydra.usdrt_delegate-7.5.1] startup
[3.355s] [ext: omni.hydra.scene_delegate-0.3.3] startup
[3.359s] [ext: omni.usd-1.12.2] startup
[3.394s] [ext: omni.kit.property.adapter.core-1.0.1] startup
[3.397s] [ext: omni.kit.property.adapter.usd-1.0.1] startup
[3.399s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[3.400s] [ext: omni.hydra.rtx.shadercache.vulkan-1.0.0] startup
[3.401s] [ext: omni.kit.property.adapter.fabric-1.0.1] startup
[3.403s] [ext: omni.kit.widget.prompt-1.0.7] startup
[3.405s] [ext: omni.kit.collaboration.channel_manager-1.0.12] startup
[3.407s] [ext: omni.kit.widget.searchable_combobox-1.0.6] startup
[3.408s] [ext: omni.kit.notification_manager-1.0.9] startup
[3.411s] [ext: omni.kit.clipboard-1.0.4] startup
[3.412s] [ext: omni.kit.usd.layers-2.1.36] startup
[3.423s] [ext: omni.kit.widget.settings-1.2.2] startup
[3.425s] [ext: omni.kit.helper.file_utils-0.1.8] startup
[3.428s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[3.429s] [ext: omni.kit.widget.filebrowser-2.10.51] startup
[3.436s] [ext: omni.kit.search_core-1.0.5] startup
[3.437s] [ext: omni.kit.widget.options_menu-1.1.6] startup
[3.442s] [ext: omni.kit.menu.core-1.0.4] startup
[3.443s] [ext: omni.kit.widget.path_field-2.0.10] startup
[3.444s] [ext: omni.kit.widget.search_delegate-1.0.5] startup
[3.446s] [ext: omni.kit.widget.options_button-1.0.3] startup
[3.447s] [ext: omni.kit.widget.context_menu-1.2.2] startup
[3.448s] [ext: omni.ui.scene-1.10.3] startup
[3.453s] [ext: omni.kit.widget.browser_bar-2.0.10] startup
[3.455s] [ext: omni.kit.menu.utils-1.5.27] startup
[3.465s] [ext: omni.kit.window.filepicker-2.10.40] startup
[3.485s] [ext: omni.kit.collaboration.presence_layer-1.0.9] startup
[3.488s] [ext: omni.kit.context_menu-1.8.1] startup
[3.490s] [ext: omni.kit.window.preferences-1.6.0] startup
[3.498s] [ext: omni.kit.widget.live_session_management.ui-1.0.1] startup
[3.501s] [ext: omni.volume-0.5.0] startup
[3.505s] [ext: omni.ujitso.processor.texture-1.0.0] startup
[3.506s] [ext: omni.kit.window.file_exporter-1.0.30] startup
[3.508s] [ext: omni.kit.widget.searchfield-1.1.8] startup
[3.510s] [ext: omni.kit.widget.filter-1.1.4] startup
[3.511s] [ext: omni.kit.hotkeys.core-1.3.5] startup
[3.512s] [ext: omni.ujitso.client-0.0.0] startup
[3.513s] [ext: omni.kit.widget.stage-2.11.2] startup
[3.526s] [ext: omni.kit.widget.live_session_management-1.2.20] startup
[3.529s] [ext: omni.hydra.rtx-1.0.0] startup
[3.543s] [ext: omni.kit.stage_template.core-1.1.22] startup
[3.544s] [ext: omni.kit.window.content_browser_registry-0.0.6] startup
[3.545s] [ext: omni.inspect-1.0.1] startup
[3.548s] [ext: omni.kit.window.file_importer-1.1.12] startup
[3.549s] [ext: omni.kit.widget.highlight_label-1.0.2] startup
[3.551s] [ext: omni.kit.window.file-1.3.54] startup
[3.555s] [ext: omni.kit.window.drop_support-1.0.3] startup
[3.555s] [ext: omni.graph.tools-1.79.0] startup
[3.588s] [ext: omni.graph.core-2.179.2] startup
[3.591s] [ext: omni.kit.window.property-1.11.3] startup
[3.595s] [ext: omni.kit.window.content_browser-2.9.18] startup
[3.611s] [ext: omni.graph-1.140.0] startup
[3.767s] [ext: omni.graph.image.core-0.4.5] startup
[3.769s] [ext: omni.kit.property.usd-4.2.8] startup
[3.782s] [ext: omni.debugdraw-0.1.3] startup
[3.786s] [ext: omni.kvdb-106.1.9] startup
[3.788s] [ext: omni.kit.material.library-1.5.6] startup
[3.797s] [ext: omni.kit.hydra_texture-1.3.9] startup
[3.801s] [ext: omni.kit.viewport.legacy_gizmos-1.1.0] startup
[3.803s] [ext: omni.kit.raycast.query-1.0.5] startup
[3.808s] [ext: omni.hydra.engine.stats-1.0.2] startup
[3.810s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[3.812s] [ext: omni.kit.widget.viewport-106.1.0] startup
[3.818s] [ext: omni.kit.viewport.registry-104.0.6] startup
[3.820s] [ext: omni.kit.widget.graph-1.12.15] startup
[3.828s] [ext: omni.kit.viewport.window-107.0.6] startup
[3.851s] [ext: omni.kit.window.cursor-1.1.2] startup
[3.853s] [ext: omni.ui_query-1.1.4] startup
[3.855s] [ext: omni.kit.window.extensions-1.4.11] startup
[3.863s] [ext: omni.kit.viewport.utility-1.0.17] startup
[3.865s] [ext: omni.graph.action_core-1.1.6] startup
[3.868s] [ext: omni.kit.viewport.menubar.core-106.1.0] startup
[3.887s] [ext: omni.graph.ui-1.70.2] startup
[3.913s] [ext: omni.graph.image.nodes-1.1.0] startup
[3.917s] [ext: omni.kit.graph.usd.commands-1.3.1] startup
[3.918s] [ext: omni.convexdecomposition-106.1.9] startup
[3.923s] [ext: omni.graph.action_nodes-1.24.0] startup
[3.928s] [ext: omni.localcache-106.1.9] startup
[3.932s] [ext: omni.syntheticdata-0.6.9] startup
[3.976s] [ext: omni.warp.core-1.2.1] startup
[4.110s] [ext: omni.graph.nodes-1.146.1] startup
[4.121s] [ext: omni.graph.action-1.102.1] startup
[4.123s] [ext: omni.physx.foundation-106.1.9] startup
[4.126s] [ext: omni.kit.graph.delegate.default-1.2.2] startup
[4.127s] [ext: omni.usdphysics-106.1.9] startup
[4.130s] [ext: omni.warp-1.2.1] startup
[4.137s] [ext: omni.kit.widget.material_preview-1.0.16] startup
[4.142s] [ext: omni.kit.graph.editor.core-1.5.3] startup
[4.147s] [ext: omni.physx.cooking-106.1.9] startup
[4.161s] [ext: omni.kit.window.material_graph-1.8.18] startup
[4.253s] [ext: omni.physx-106.1.9] startup
[4.267s] [ext: omni.kit.numpy.common-0.1.2] startup
[4.270s] [ext: omni.isaac.version-1.1.0] startup
[4.271s] [ext: omni.kit.primitive.mesh-1.0.17] startup
[4.276s] [ext: omni.kit.manipulator.viewport-107.0.0] startup
[4.278s] [ext: omni.physx.stageupdate-106.1.9] startup
[4.283s] [ext: omni.physics.tensors-106.1.9] startup
[4.294s] [ext: omni.isaac.nucleus-0.3.1] startup
[4.297s] [ext: omni.kit.stage_templates-1.2.5] startup
[4.304s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[4.311s] [ext: omni.isaac.dynamic_control-1.3.8] startup
[4.326s] [ext: omni.physx.tensors-106.1.9] startup
[4.331s] [ext: omni.graph.scriptnode-1.20.1] startup
[4.332s] [ext: omni.kit.ui_test-1.3.2] startup
[4.338s] [ext: omni.isaac.core-3.19.5] startup
[5.790s] [ext: omni.replicator.core-1.11.20] startup
2024-09-29 18:03:53 [5,751ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
Warp 1.2.1 initialized:
   CUDA Toolkit 11.8, Driver 12.4
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA GeForce RTX 3090" (24 GiB, sm_86, mempool enabled)
   Kernel cache:
     /root/.cache/warp/1.2.1
[5.980s] [ext: omni.isaac.ui-0.16.0] startup
[5.994s] [ext: omni.isaac.lula-3.0.1] startup
[6.007s] [ext: omni.isaac.surface_gripper-1.0.1] startup
[6.012s] [ext: omni.isaac.core_nodes-1.16.3] startup
[6.044s] [ext: omni.isaac.motion_generation-7.1.0] startup
[6.076s] [ext: omni.isaac.manipulators-2.1.0] startup
[6.091s] [ext: omni.isaac.franka-0.4.1] startup
[6.100s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[6.108s] [ext: omni.kit.graph.delegate.modern-1.10.6] startup
[6.122s] [ext: omni.isaac.cortex-0.3.9] startup
[6.124s] [ext: omni.graph.window.core-1.113.1] startup
[6.166s] [ext: omni.isaac.wheeled_robots-2.3.3] startup
[6.181s] [ext: omni.isaac.cortex.sample_behaviors-1.0.6] startup
[6.187s] [ext: omni.graph.window.generic-1.26.0] startup
[6.205s] [ext: omni.isaac.debug_draw-1.1.0] startup
[6.212s] [ext: omni.isaac.block_world-1.0.0] startup
[6.220s] [ext: omni.isaac.menu-0.7.3] startup
[6.228s] [ext: omni.isaac.occupancy_map-1.0.3] startup
[6.245s] [ext: omni.kit.viewport.actions-106.0.2] startup
[6.259s] [ext: omni.anim.retarget.core-106.1.2] startup
[6.278s] [ext: omni.anim.asset-106.1.0] startup
[6.282s] [ext: omni.kit.viewport.menubar.display-107.0.2] startup
[6.289s] [ext: omni.anim.graph.core-106.1.2] startup
[6.326s] [ext: omni.importer.mjcf-1.1.1] startup
[6.337s] [ext: omni.anim.skelJoint-106.1.2] startup
2024-09-29 18:03:53 [6,274ms] [Warning] [omni.graph.core._impl._registration.register_python_ogn] Python import process in omni.anim.graph.core failed - 'NoneType' object is not subscriptable. Aborting Python node registration
[6.348s] [ext: omni.kit.widget.live-2.1.8] startup
[6.352s] [ext: omni.kit.actions.window-1.1.1] startup
[6.357s] [ext: omni.kit.ui.actions-1.0.2] startup
[6.359s] [ext: omni.kit.widget.cache_indicator-2.0.10] startup
[6.423s] [ext: omni.kit.hotkeys.window-1.4.5] startup
2024-09-29 18:03:53 [6,357ms] [Warning] [omni.kit.widget.cache_indicator.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
[6.433s] [ext: omni.kit.menu.common-1.1.7] startup
[6.437s] [ext: omni.kit.widget.stage_icons-1.0.5] startup
[6.441s] [ext: omni.kit.selection-0.1.4] startup
[6.443s] [ext: omni.isaac.kit-1.13.2] startup
[6.444s] [ext: omni.kit.window.stage-2.5.10] startup
[6.449s] [ext: omni.kit.menu.edit-1.1.24] startup
[6.457s] [ext: omni.kit.menu.file-1.1.14] startup
[6.462s] [ext: omni.kit.menu.stage-1.2.5] startup
[6.464s] [ext: omni.kit.property.audio-1.0.14] startup
[6.470s] [ext: omni.kit.property.geometry-1.3.1] startup
[6.476s] [ext: omni.sensors.nv.common-1.2.2-isaac-1] startup
[6.488s] [ext: omni.kit.property.camera-1.0.8] startup
[6.491s] [ext: omni.kit.property.material-1.10.8] startup
[6.501s] [ext: omni.kit.property.light-1.0.10] startup
[6.504s] [ext: omni.sensors.nv.materials-1.2.1-isaac-1] startup
[6.510s] [ext: omni.kit.profiler.window-2.2.3] startup
[6.522s] [ext: omni.sensors.nv.wpm-1.2.1-isaac-1] startup
[6.524s] [ext: omni.sensors.nv.ids-1.1.0-isaac-1] startup
[6.527s] [ext: omni.hydra.scene_api-0.1.2] startup
[6.532s] [ext: omni.sensors.tiled-0.0.6] startup
[6.533s] [ext: omni.sensors.nv.radar-1.2.1-isaac-1] startup
[6.539s] [ext: omni.sensors.nv.lidar-1.2.2-isaac-1] startup
[6.543s] [ext: omni.isaac.range_sensor-3.1.2] startup
[6.563s] [ext: omni.kit.property.transform-1.5.9] startup
[6.572s] [ext: omni.kit.property.render-1.1.2] startup
[6.574s] [ext: omni.isaac.sensor-12.9.1] startup
[6.666s] [ext: omni.kit.widget.layers-1.8.0] startup
[6.699s] [ext: omni.kit.property.bundle-1.3.1] startup
[6.702s] [ext: omni.kit.property.isaac-0.2.3] startup
[6.704s] [ext: omni.isaac.quadruped-2.0.1] startup
[6.720s] [ext: omni.kit.property.layer-1.1.8] startup
[6.725s] [ext: omni.kit.stage_column.payload-2.0.0] startup
[6.728s] [ext: omni.kit.widget.toolbar-1.7.2] startup
[6.738s] [ext: omni.kit.manipulator.transform-104.7.4] startup
[6.742s] [ext: omni.physx.commands-106.1.9] startup
[6.747s] [ext: omni.usdphysics.ui-106.1.9] startup
[6.769s] [ext: omni.kit.manipulator.tool.snap-1.5.11] startup
[6.777s] [ext: omni.isaac.scene_blox-0.1.2] startup
[6.778s] [ext: omni.kit.manipulator.selector-1.1.1] startup
[6.781s] [ext: omni.physx.ui-106.1.9] startup
[6.815s] [ext: omni.importer.urdf-1.14.1] startup
[6.868s] [ext: omni.kit.viewport.manipulator.transform-107.0.2] startup
[6.870s] [ext: omni.physx.demos-106.1.9] startup
[6.900s] [ext: omni.kit.property.physx-106.1.9] startup
[6.975s] [ext: omni.kit.widget.calendar-1.0.8] startup
[6.977s] [ext: omni.kit.manipulator.prim.core-107.0.4] startup
[6.988s] [ext: omni.fabric.commands-1.1.5] startup
[6.993s] [ext: omni.physx.vehicle-106.1.9] startup
[7.018s] [ext: omni.kit.widget.extended_searchfield-1.0.28] startup
[7.031s] [ext: omni.kit.manipulator.prim.fabric-107.0.3] startup
[7.035s] [ext: omni.kit.manipulator.prim.usd-107.0.2] startup
[7.037s] [ext: omni.kit.widget.timeline-105.0.1] startup
[7.041s] [ext: omni.kit.window.commands-0.2.6] startup
[7.044s] [ext: omni.physx.camera-106.1.9] startup
[7.057s] [ext: omni.kit.manipulator.camera-105.0.5] startup
[7.062s] [ext: omni.kit.viewport.menubar.camera-107.0.2] startup
[7.070s] [ext: omni.kit.manipulator.prim-107.0.0] startup
[7.070s] [ext: omni.kit.manipulator.selection-106.0.1] startup
[7.072s] [ext: omni.kit.window.toolbar-1.6.1] startup
[7.075s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[7.078s] [ext: omni.kit.viewport.menubar.render-106.1.1] startup
[7.082s] [ext: omni.kit.viewport.menubar.settings-107.0.3] startup
[7.089s] [ext: omni.physx.cct-106.1.9] startup
[7.102s] [ext: omni.physx.graph-106.1.9] startup
[7.124s] [ext: omni.physx.supportui-106.1.9] startup
[7.143s] [ext: omni.physx.telemetry-106.1.9] startup
[7.147s] [ext: omni.graph.ui_nodes-1.26.0] startup
[7.152s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[7.153s] [ext: omni.isaac.universal_robots-0.3.5] startup
[7.156s] [ext: omni.kit.window.console-0.2.13] startup
[7.168s] [ext: omni.kit.window.script_editor-1.7.6] startup
[7.172s] [ext: omni.kit.menu.create-1.0.16] startup
[7.176s] [ext: omni.graph.bundle.action-2.4.1] startup
[7.179s] [ext: omni.physx.bundle-106.1.9] startup
[7.180s] [ext: omni.rtx.window.settings-0.6.17] startup
[7.190s] [ext: omni.kit.window.status_bar-0.1.7] startup
[7.194s] [ext: omni.kit.window.title-1.1.5] startup
[7.197s] [ext: omni.replicator.isaac-1.15.2] startup
[7.229s] [ext: omni.replicator.replicator_yaml-2.0.6] startup
[7.240s] [ext: omni.rtx.settings.core-0.6.3] startup
[7.251s] [ext: omni.scene.optimizer.core-106.1.1] startup
[7.332s] [ext: omni.kit.viewport.menubar.lighting-106.0.2] startup
[7.340s] [ext: semantics.schema.editor-0.3.8] startup
[7.344s] [ext: semantics.schema.property-1.0.4] startup
[7.346s] [ext: omni.kit.viewport.rtx-104.0.1] startup
[7.347s] [ext: omni.isaac.utils-1.0.1] startup
[7.351s] [ext: omni.isaac.cloner-0.8.1] startup
[7.355s] [ext: omni.graph.window.action-1.28.0] startup
[7.363s] [ext: omni.kit.window.stats-0.1.6] startup
[7.366s] [ext: omni.isaac.sim.python-4.2.0] startup
[7.367s] Simulation App Starting
[10.015s] app ready
[11.221s] Simulation App Startup Complete
2024-09-29 18:03:58 [11,459ms] [Warning] [srv_program] Starting
2024-09-29 18:03:58 [11,461ms] [Warning] [srv_program] Warming up...
2024-09-29 18:03:58 [11,502ms] [Warning] [srv_program] Creating...
2024-09-29 18:03:58 [11,506ms] [Warning] [srv_program] Loading...
2024-09-29 18:03:58 [11,562ms] [Warning] [omni.usd] Unexpected reference count of 2 for UsdStage 'anon:0x1727af80:World0.usd' while being closed in UsdContext (this may indicate it is still resident in memory).
2024-09-29 18:04:21 [33,660ms] [Warning] [srv_program] Warming up...
2024-09-29 18:04:32 [44,899ms] [Warning] [omni.syntheticdata.plugin] OgnSdPostRenderVarToHost : rendervar copy from texture directly to host buffer is counter-performant. Please use copy from texture to device buffer first.
2024-09-29 18:04:42 [55,631ms] [Warning] [srv_program] On trigger...
2024-09-29 18:04:44 [57,451ms] [Warning] [carb] Client omni.stageupdate.plugin has acquired [omni::hydra::IOmniHydra v2.0] 100 times. Consider accessing this interface with carb::getCachedInterface() (Performance warning)
2024-09-29 18:05:22 [94,749ms] [Warning] [srv_program] Running...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_05/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_06/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_07/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_08/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_09/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_10/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_11/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_12/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_13/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_14/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_15/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_16/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_17/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_18/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_19/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:31 [103,997ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_20/Ref/agg_persons>) has no variant  in the variant set agg_phones_l. Skipping...
2024-09-29 18:05:33 [106,218ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_05/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,218ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_06/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,218ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_07/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,219ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_08/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,219ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_10/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,219ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_11/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,219ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_12/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,219ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_14/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,220ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_15/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,220ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_16/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,220ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_17/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,220ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_18/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,221ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_19/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:33 [106,221ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_20/Ref/agg_persons>) has no variant  in the variant set agg_phones_r. Skipping...
2024-09-29 18:05:35 [107,848ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_05/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,848ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_06/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,848ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_07/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_08/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_09/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_10/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_11/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_12/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,849ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_13/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,850ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_14/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,850ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_15/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,850ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_16/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,850ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_17/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,850ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_18/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
2024-09-29 18:05:35 [107,851ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_19/Ref/agg_persons>) has no variant  in the variant set agg_glasses. Skipping...
/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

2024-09-29 18:05:36 [108,772ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_05/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,773ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_06/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,773ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_07/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,773ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_08/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,773ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_09/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,774ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_10/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,774ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_11/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,774ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_12/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,775ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_13/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,775ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_14/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,775ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_15/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,775ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_16/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,776ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_17/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,776ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_18/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,776ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_19/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
2024-09-29 18:05:36 [108,777ms] [Warning] [omni.replicator.core.ogn.python._impl.nodes.OgnSetVariant] Usd.Prim(</Replicator/Ref_Xform_20/Ref/agg_persons>) has no variant  in the variant set agg_helmets. Skipping...
/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

/builds/omniverse/physics/physx/source/foundation/FdFoundation.cpp (247) : invalid operation : Foundation object exists already. Only one instance per process can be created.

2024-09-29 18:09:06 [318,740ms] [Warning] [omni.hydra] Parameter 'diffuse_tint' of shade node 'usd::rtx_scope0::/Replicator/Ref_Xform/Ref/Looks/Concrete_Polished/Shader::::Z6Ahttp_3A_2F::Z59omniverse_2Dcontent_2Dproduction_2Es3_2Eus_2Dwest_2D2_2Eamazonaws_2Ecom::Materials::vMaterials_2::Concrete::Concrete_Polished::Concrete_Polished((3511))' not available in the MDL representation.
2024-09-29 18:09:06 [319,187ms] [Warning] [omni.hydra] Parameter 'diffuse_tint' of shade node 'usd::rtx_scope0::/Replicator/Ref_Xform/Ref/Looks/Concrete_Precast/Shader::::Z6Ahttp_3A_2F::Z59omniverse_2Dcontent_2Dproduction_2Es3_2Eus_2Dwest_2D2_2Eamazonaws_2Ecom::Materials::vMaterials_2::Concrete::Concrete_Precast::Concrete_Precast((3512))' not available in the MDL representation.
2024-09-29 18:09:14 [327,462ms] [Warning] [omni.hydra] Parameter 'diffuse_tint' of shade node 'usd::rtx_scope0::/Replicator/Ref_Xform/Ref/Looks/Spongy_Concrete_Weathered_Mossy_Dark_Dirt/Shader::::Z6Ahttp_3A_2F::Z59omniverse_2Dcontent_2Dproduction_2Es3_2Eus_2Dwest_2D2_2Eamazonaws_2Ecom::Materials::vMaterials_2::Concrete::Spongy_Concrete_Weathered_Mossy::Spongy_Concrete_Weathered_Mossy_Dark_Dirt((3579))' not available in the MDL representation.
2024-09-29 18:09:17 [330,608ms] [Warning] [omni.hydra] Parameter 'diffuse_tint' of shade node 'usd::rtx_scope0::/Replicator/Ref_Xform/Ref/Looks/Concrete_Floor_Damage/Shader::::Z6Ahttp_3A_2F::Z59omniverse_2Dcontent_2Dproduction_2Es3_2Eus_2Dwest_2D2_2Eamazonaws_2Ecom::Materials::vMaterials_2::Concrete::Concrete_Floor_Damage::Concrete_Floor_Damage((3595))' not available in the MDL representation.
2024-09-29 18:09:19 [331,710ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /Replicator/SampledAssets/Population_66530c92/Ref_Xform_0/Ref.proto_Section1_id1 that has not been populated
2024-09-29 18:10:22 [395,364ms] [Warning] [omni.replicator.core.scripts.orchestrator] Maximum loading time reached while waiting for assets to load. Asset loading may not have completed.
                Maximum asset loading time can be configured by setting the `/exts/omni.replicator.core/maxAssetLoadingTime` setting.

Same here. Looking in the code, it seems 30 seconds is the limit if no other value is set. Not sure about the meaning though. Apart from that, you could modify it like any other carb setting.

You can set a new maximum asset loading time in script using carb.settings.get_settings().set("/exts/omni.replicator.core/maxAssetLoadingTime", <new max time>) or on startup by adding the flag --/exts/omni.replicator.core/maxAssetLoadingTime=<new max time> to the startup command (eg. isaacsim.sh --/exts/omni.replicator.core/maxAssetLoadingTime=<new max time>)