Dear Team,
We are testing 4.0.0 in order to switch to using it, as it solves many of the problems we have been previously reported. Thank you and the team for such amazing work.
That said, unfortunately, our YAML scenarios that are fine on 2023.1.1 do not even load on 4.0.0. Both the Isaac Sim Window App and Container (on Ubuntu) are stuck while parsing the YAML scenarios. It doesn’t even reach loading any of the assets. It seems like the randomizer graph is blowing up to infinity as the CPU is going to 100% on a single core for the kit process. We have tried waiting 24 hours to see if it will load but it won’t. It seems like adding a certain amount of randomizer actions inside the on_frame causes the issue.
Below you can find:
- reproduction scenario,
- reproduction command,
- log
In the scenario, removing uniformm0001_instance4, uniformm0001_instance3, uniformm0001_instance2 “fixes” the problem. Basically there seems to be a limit to how many groups one can have. In the previous versions of Isaac that wasn’t the problem as we have 100s of code blocks like that. We need this sorted out quick.
Reproduction Scenario
# Default scene settings. This script default is 1 (IsaacSim), so if you change
# the meters per unit or up axis, you need to alter the coordinates and rotations.
# IsaacSim default is 1, Code is 0.01
stage_unit_setting:
settings.set_stage_meters_per_unit:
meters_per_unit: 1
# IsaacSim default is "Z"", Code is "Y"
stage_up_axis_setting:
settings.set_stage_up_axis:
up_axis: Z
# Enable Path Tracing
path_tracing:
settings.set_render_pathtraced:
samples_per_pixel: 32
# Load the stage
load_stage:
create.from_usd:
usd: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/full_warehouse.usd
# Select the objects in the stage
forklift:
get.prim_at_path:
path: /Replicator/Ref_Xform/Ref/forklift
floor:
get.prims:
semantics: [[class, floor]]
racks_and_walls:
get.prims:
semantics: [[class, rack], [class, wall]]
default_light:
get.prim_at_path:
path: /Environment/defaultLight
light:
create.group:
items:
- /Replicator/Ref_Xform/Ref/RectLight
- /Replicator/Ref_Xform/Ref/RectLight_01
- /Replicator/Ref_Xform/Ref/RectLight_02
- /Replicator/Ref_Xform/Ref/RectLight_03
- /Replicator/Ref_Xform/Ref/RectLight_04
- /Replicator/Ref_Xform/Ref/RectLight_05
- /Replicator/Ref_Xform/Ref/RectLight_06
camera:
create.camera:
position: [0, 0, 0]
rotation: [0, 0, 0]
# Create output
render_product:
create.render_product:
camera: camera
resolution: [1280, 720]
writer:
writers.get:
name: BasicWriter
init_params:
rgb: True
bounding_box_2d_tight: True
semantic_filter_predicate: class:sign|cone|person
output_dir: /workspaces/Surveily.Datasets.Generator.Omniverse/.out/Issues/stuck
#output_dir: /home/wojtek/stuck
writer_attach:
writer.attach:
render_products: render_product
# Load businessf0002 animations
businessf0002_load_animations:
create.from_dir:
recursive: True
semantics: [[animation, businessf0002]]
dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Motion/
businessf0002_animations:
get.prims:
semantics: [[animation, businessf0002]]
# Load partym0001 animations
partym0001_load_animations:
create.from_dir:
recursive: True
semantics: [[animation, partym0001]]
dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Party_M_0001/Motion/
partym0001_animations:
get.prims:
semantics: [[animation, partym0001]]
# Load uniformf0001 animations
uniformf0001_load_animations:
create.from_dir:
recursive: True
semantics: [[animation, uniformf0001]]
dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_F_0001/Motion/
uniformf0001_animations:
get.prims:
semantics: [[animation, uniformf0001]]
# Load uniformm0001 animations
uniformm0001_load_animations:
create.from_dir:
recursive: True
semantics: [[animation, uniformm0001]]
dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_M_0001/Motion/
uniformm0001_animations:
get.prims:
semantics: [[animation, uniformm0001]]
# Load worker animations
worker_load_animations:
create.from_dir:
recursive: True
semantics: [[animation, worker]]
dir_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/Worker/Motions/
worker_animations:
get.skelanimation:
semantics: [[animation, worker]]
# Set the trigger as on_frame
trigger:
trigger.on_frame:
max_execs: 20
rt_subframes: 5
# When the trigger executes, run the following logic
with_trigger:
with.trigger:
# Disable default light
with.default_light:
modify.visibility:
value: False
# Set sun intensity
sun_intensity:
with.light:
modify.attribute:
name: inputs:intensity
value:
distribution.uniform:
lower: 500
upper: 5000
# Set sun temperature
sun_temperature_on:
with.light:
modify.attribute:
name: inputs:enableColorTemperature
value: True
# Set sun temperature
sun_temperature:
with.light:
modify.attribute:
name: inputs:colorTemperature
value:
distribution.uniform:
lower: 4000
upper: 7000
# move the camera around and rotate
with.camera:
modify.pose:
position:
distribution.uniform:
lower: [-23.6, -3, 2.5]
upper: [3.03, 3, 7.5]
rotation:
distribution.uniform:
lower: [0, -30, -120]
upper: [0, -10, -60]
# Create and scatter businessf0002
businessf0002_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Business_F_0002/Actor/business-f-0002/business-f-0002.usd
size: 5
mode: reference
with.businessf0002_instance:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: businessf0002_animations
randomizer.scatter_2d:
seed: 1
surface_prims: floor
check_for_collisions: True
# Create and scatter partym0001
partym0001_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Party_M_0001/Actor/party-m-0001/party-m-0001.usd
size: 5
mode: reference
with.partym0001_instance:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: partym0001_animations
randomizer.scatter_2d:
seed: 2
surface_prims: floor
check_for_collisions: True
# Create and scatter uniformf0001
uniformf0001_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_F_0001/Actor/uniform_f_0001/uniform_f_0001.usd
size: 5
mode: reference
with.uniformf0001_instance:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: uniformf0001_animations
randomizer.scatter_2d:
seed: 3
surface_prims: floor
check_for_collisions: True
# Create and scatter uniformm0001
uniformm0001_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_M_0001/Actor/uniform_m_0001/uniform_m_0001.usd
size: 5
mode: reference
with.uniformm0001_instance:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: uniformm0001_animations
randomizer.scatter_2d:
seed: 4
surface_prims: floor
check_for_collisions: True
# Create and scatter uniformm00012
uniformm0001_instance2:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_M_0001/Actor/uniform_m_0001/uniform_m_0001.usd
size: 5
mode: reference
with.uniformm0001_instance2:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: uniformm0001_animations
randomizer.scatter_2d:
seed: 4
surface_prims: floor
check_for_collisions: True
# Create and scatter uniformm00013
uniformm0001_instance3:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_M_0001/Actor/uniform_m_0001/uniform_m_0001.usd
size: 5
mode: reference
with.uniformm0001_instance3:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: uniformm0001_animations
randomizer.scatter_2d:
seed: 4
surface_prims: floor
check_for_collisions: True
# Create and scatter uniformm00014
uniformm0001_instance4:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/ActorCore/Uniform_M_0001/Actor/uniform_m_0001/uniform_m_0001.usd
size: 5
mode: reference
with.uniformm0001_instance4:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
modify.animation:
values:
distribution.sequence:
items: uniformm0001_animations
randomizer.scatter_2d:
seed: 4
surface_prims: floor
check_for_collisions: True
# Create and scatter worker
worker_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/NVIDIA/Assets/Characters/Reallusion/Worker/Props/Worker.usd
size: 5
mode: reference
with.worker_instance:
modify.pose:
scale: 0.01
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
modify.semantics:
semantics: [[class, person]]
# modify.animation:
# values:
# distribution.sequence:
# items: worker_animations
randomizer.scatter_2d:
seed: 5
surface_prims: floor
check_for_collisions: True
# create and scatter cones
cone_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_TrafficCone.usd
size: 20
mode: scene_instance
with.cone_instance:
modify.pose:
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
randomizer.scatter_2d:
seed: 101
surface_prims: floor
no_coll_prims: racks_and_walls
check_for_collisions: True
# create and scatter wet floor signs
wet_floor_sign_instance:
randomizer.instantiate:
paths:
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2023.1.1/Isaac/Environments/Simple_Warehouse/Props/S_WetFloorSign.usd
size: 20
mode: scene_instance
with.wet_floor_sign_instance:
modify.pose:
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
randomizer.scatter_2d:
seed: 102
surface_prims: floor
no_coll_prims: racks_and_walls
check_for_collisions: True
# move the forklift around and rotate
with.forklift:
modify.pose:
rotation:
distribution.uniform:
lower: [0, 0, -180]
upper: [0, 0, 180]
# randomizer.scatter_2d:
# seed: 0
# surface_prims: floor
# no_coll_prims: racks_and_walls
# check_for_collisions: True
# Randomize floor
with.floor:
modify.material:
value:
- /Replicator/Ref_Xform/Ref/SM_floor27/Looks/MI_Floor_02b
- /Replicator/Ref_Xform/Ref/SM_WallA_3M42/SM_WallA_6M/Looks/MI_WallA_01
Reproduction Command
#!/bin/bash
set -e
FILE=$1
echo $FILE
if [ -f "$FILE" ]; then
/isaac-sim/runheadless.native.sh --no-window \
--/app/viewport/grid/enabled=false \
--/app/viewport/defaults/guide/grid/visible=false \
--/omni/replicator/replicatorYaml/yamlPath=$FILE
exit $?
else
echo "File not found: $FILE"
exit 1
fi
Log
The NVIDIA Omniverse License Agreement (EULA) must be accepted before
Omniverse Kit can start. The license terms for this product can be viewed at
https://docs.omniverse.nvidia.com/app_isaacsim/common/NVIDIA_Omniverse_License_Agreement.html
Omniverse Software collects installation and configuration details about your software, hardware, and network
configuration (e.g., version of operating system, applications installed, type of hardware, network speed, IP
address) based on our legitimate interest in improving your experience. To improve performance, troubleshooting
and diagnostic purposes of our software, we also collect session behavior, error and crash logs.
Data Collection in container mode is completely anonymous unless specified. You may opt-out of this collection
anytime by not setting the OMNI_ENV_PRIVACY_CONSENT environment variable.
To opt-in set the OMNI_ENV_PRIVACY_CONSENT environment variable when running the container. Set the
OMNI_ENV_PRIVACY_USERID environment variable tag the telemetry data with a user ID.
Loading user config located at: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/user.config.json'
[Info] [carb] Logging to file: /root/.nvidia-omniverse/logs/Kit/Isaac-Sim/4.0/kit_20240704_133835.log
2024-07-04 13:38:35 [1ms] [Warning] [carb.crashreporter-breakpad.plugin] Failed to parse toml file '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/b1fb1dda-d815-49dd-c713c5b9-b7a01f5c.dmp.toml': /root/.local/share/ov/data/Kit/Isaac-Sim/4.0/b1fb1dda-d815-49dd-c713c5b9-b7a01f5c.dmp.toml could not be opened for parsing
2024-07-04 13:38:35 [1ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/b1fb1dda-d815-49dd-c713c5b9-b7a01f5c.dmp'
2024-07-04 13:38:35 [1ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/566d147c-a3ef-4b9c-1d8b74b4-f4e53ce6.dmp.zip'
2024-07-04 13:38:35 [2ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/32e7b4f7-856e-4ac8-d44f8a9a-cb31077e.dmp.zip'
2024-07-04 13:38:35 [2ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/c978f5e5-9abd-45aa-763052b7-b485711a.dmp.zip'
2024-07-04 13:38:35 [2ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/6bcaf074-cf4f-4d55-82818798-a7c72cde.dmp.zip'
2024-07-04 13:38:35 [3ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/e268bb75-ddfd-410b-26f1bb80-f0d713ac.dmp.zip'
2024-07-04 13:38:35 [3ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: '/root/.local/share/ov/data/Kit/Isaac-Sim/4.0/90449153-9f7f-41fb-56f54c84-6b9fefaf.dmp.zip'
[0.196s] [ext: omni.kit.async_engine-0.0.0] startup
[0.804s] [ext: omni.stats-0.0.0] startup
[0.805s] [ext: omni.client-1.1.0] startup
[0.828s] [ext: omni.datastore-0.0.0] startup
[0.828s] [ext: omni.blobkey-1.1.0] startup
[0.828s] [ext: omni.ujitso.default-1.0.0] startup
[0.829s] [ext: omni.hsscclient-0.0.0] startup
[0.829s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[0.830s] [ext: omni.assets.plugins-0.0.0] startup
[0.831s] [ext: omni.gpu_foundation-0.0.0] startup
[0.841s] [ext: carb.windowing.plugins-1.0.0] startup
[0.870s] [ext: omni.kit.renderer.init-0.0.0] startup
|---------------------------------------------------------------------------------------------|
| Driver Version: 550.90.07 | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
| | | | | | Bus-ID | |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 3060 Laptop.. | Yes: 0 | | 6144 MB | 10de | 0 |
| | | | | | 2520 | 1ddb812e.. |
| | | | | | 1 | |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 6.8.0-36-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101011 (1.21.1.11)
| Processor: AMD Ryzen 7 6800HS with Radeon Graphics | Cores: 8 | Logical: 16
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 39434 | Free Memory: 31546
| Total Page/Swap (MB): 8191 | Free Page/Swap: 8191
|---------------------------------------------------------------------------------------------|
2024-07-04 13:38:37 [1,219ms] [Warning] [gpu.foundation.plugin] IOMMU is enabled.
[1.409s] [ext: omni.kit.pipapi-0.0.0] startup
[1.411s] [ext: omni.kit.pip_archive-0.0.0] startup
[1.411s] [ext: omni.pip.compute-1.3.1] startup
[1.411s] [ext: omni.mtlx-0.1.0] startup
[1.412s] [ext: omni.usd.config-1.0.3] startup
[1.416s] [ext: omni.gpucompute.plugins-0.0.0] startup
[1.416s] [ext: omni.usd.libs-1.0.0] startup
[1.485s] [ext: omni.pip.cloud-1.1.3] startup
[1.488s] [ext: omni.isaac.core_archive-2.3.0] startup
[1.488s] [ext: omni.kit.telemetry-0.5.0] startup
[1.516s] [ext: omni.kit.loop-isaac-1.2.0] startup
[1.517s] [ext: omni.kit.test-0.0.0] startup
[1.571s] [ext: omni.isaac.ml_archive-2.0.1] startup
[1.571s] [ext: omni.appwindow-1.1.8] startup
[1.573s] [ext: omni.kit.renderer.core-1.0.1] startup
[1.584s] [ext: omni.kit.renderer.capture-0.0.0] startup
[1.586s] [ext: omni.kit.renderer.imgui-1.0.1] startup
[1.660s] [ext: omni.ui-2.23.11] startup
[1.670s] [ext: omni.kit.mainwindow-1.0.3] startup
[1.671s] [ext: carb.audio-0.1.0] startup
[1.675s] [ext: omni.uiaudio-1.0.0] startup
[1.676s] [ext: omni.kit.uiapp-0.0.0] startup
[1.676s] [ext: omni.usd.schema.metrics.assembler-106.0.1] startup
[1.780s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[1.786s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[1.790s] [ext: omni.usd.schema.semantics-0.0.0] startup
[1.807s] [ext: omni.usd.schema.anim-0.0.0] startup
[1.823s] [ext: omni.usd.schema.audio-0.0.0] startup
[1.826s] [ext: omni.anim.navigation.schema-106.0.2] startup
[1.829s] [ext: omni.usd.schema.isaac-2.0.2] startup
[1.835s] [ext: omni.usd.schema.physx-106.0.16] startup
[1.878s] [ext: omni.anim.graph.schema-106.0.2] startup
[1.884s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[1.886s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[1.890s] [ext: omni.graph.exec-0.9.3] startup
[1.891s] [ext: omni.kit.actions.core-1.0.0] startup
[1.892s] [ext: omni.kit.usd_undo-0.1.8] startup
[1.892s] [ext: omni.kit.exec.core-0.13.0] startup
[1.894s] [ext: omni.kit.commands-1.4.9] startup
[1.898s] [ext: omni.usd_resolver-1.0.0] startup
[1.902s] [ext: omni.resourcemonitor-105.0.1] startup
[1.904s] [ext: omni.kit.window.popup_dialog-2.0.24] startup
[1.907s] [ext: omni.activity.core-1.0.1] startup
[1.909s] [ext: omni.usd.core-1.2.11] startup
[1.912s] [ext: omni.timeline-1.0.10] startup
[1.914s] [ext: omni.kit.widget.nucleus_connector-1.1.5] startup
[1.916s] [ext: usdrt.scenegraph-7.4.8] startup
[1.963s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[1.965s] [ext: omni.hydra.usdrt_delegate-7.4.7] startup
[1.981s] [ext: omni.hydra.scene_delegate-0.3.3] startup
[1.986s] [ext: omni.iray.libs-0.0.0] startup
[1.991s] [ext: omni.usd-1.10.39] startup
[2.025s] [ext: omni.mdl.neuraylib-0.2.4] startup
[2.026s] [ext: omni.kit.asset_converter-2.1.10] startup
[2.145s] [ext: omni.usd.schema.forcefield-106.0.16] startup
[2.150s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[2.151s] [ext: omni.kit.collaboration.channel_manager-1.0.11] startup
[2.152s] [ext: omni.kit.widget.path_field-2.0.9] startup
[2.153s] [ext: omni.kit.usd.layers-2.1.30] startup
[2.161s] [ext: omni.kit.clipboard-1.0.3] startup
[2.162s] [ext: omni.kit.widget.browser_bar-2.0.9] startup
[2.163s] [ext: omni.kit.notification_manager-1.0.8] startup
[2.165s] [ext: omni.kit.collaboration.presence_layer-1.0.8] startup
[2.166s] [ext: omni.kit.helper.file_utils-0.1.7] startup
[2.168s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[2.169s] [ext: omni.kit.widget.options_menu-1.1.4] startup
[2.171s] [ext: omni.kit.widget.filebrowser-2.3.47] startup
[2.176s] [ext: omni.kit.search_core-1.0.5] startup
[2.176s] [ext: omni.kit.widget.options_button-1.0.2] startup
[2.177s] [ext: omni.kit.widget.versioning-1.4.6] startup
[2.179s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[2.180s] [ext: omni.kit.widget.context_menu-1.2.1] startup
[2.180s] [ext: omni.convexdecomposition-106.0.16] startup
[2.184s] [ext: omni.ui.scene-1.9.3] startup
[2.189s] [ext: omni.kit.menu.utils-1.5.23] startup
[2.200s] [ext: omni.kit.window.filepicker-2.10.32] startup
[2.210s] [ext: omni.kvdb-106.0.16] startup
[2.212s] [ext: omni.physx.foundation-106.0.16] startup
[2.212s] [ext: omni.kit.widget.searchable_combobox-1.0.5] startup
[2.213s] [ext: omni.usdphysics-106.0.16] startup
[2.215s] [ext: omni.localcache-106.0.16] startup
[2.216s] [ext: omni.kit.widget.prompt-1.0.7] startup
[2.217s] [ext: omni.kit.widget.settings-1.1.0] startup
[2.219s] [ext: omni.physx.cooking-106.0.16] startup
[2.230s] [ext: omni.kit.context_menu-1.8.0] startup
[2.231s] [ext: omni.kit.window.preferences-1.5.1] startup
[2.234s] [ext: omni.kit.widget.live_session_management.ui-1.0.1] startup
[2.237s] [ext: omni.kit.widget.filter-1.1.3] startup
[2.237s] [ext: omni.inspect-1.0.1] startup
[2.239s] [ext: omni.kit.window.file_exporter-1.0.29] startup
[2.242s] [ext: omni.kit.hotkeys.core-1.3.3] startup
[2.242s] [ext: omni.kit.widget.graph-1.12.6] startup
[2.247s] [ext: omni.graph.core-2.170.0] startup
[2.250s] [ext: omni.kit.widget.stage-2.10.26] startup
[2.260s] [ext: omni.kit.widget.live_session_management-1.2.18] startup
[2.262s] [ext: omni.graph.tools-1.77.0] startup
[2.282s] [ext: omni.kit.window.extensions-1.4.7] startup
[2.287s] [ext: omni.ujitso.processor.texture-1.0.0] startup
[2.288s] [ext: omni.kit.window.file_importer-1.1.11] startup
[2.289s] [ext: omni.kit.primitive.mesh-1.0.16] startup
[2.292s] [ext: omni.volume-0.5.0] startup
[2.294s] [ext: omni.ujitso.client-0.0.0] startup
[2.295s] [ext: omni.mdl-52.0.1] startup
[2.313s] [ext: omni.kit.stage_template.core-1.1.20] startup
[2.314s] [ext: omni.kit.widget.highlight_label-1.0.1] startup
[2.314s] [ext: omni.kit.stage_templates-1.2.2] startup
[2.316s] [ext: omni.hydra.rtx-0.2.0] startup
[2.328s] [ext: omni.kit.material.library-1.4.0] startup
[2.333s] [ext: omni.hydra.engine.stats-1.0.1] startup
[2.336s] [ext: omni.kit.widget.searchfield-1.1.5] startup
[2.337s] [ext: omni.kit.window.file-1.3.52] startup
[2.339s] [ext: omni.kit.hydra_texture-1.2.6] startup
[2.344s] [ext: omni.kit.window.drop_support-1.0.1] startup
[2.344s] [ext: omni.kit.widget.viewport-106.0.2] startup
[2.348s] [ext: omni.kit.viewport.registry-104.0.6] startup
[2.348s] [ext: omni.kit.raycast.query-1.0.5] startup
[2.354s] [ext: omni.kit.window.content_browser_registry-0.0.4] startup
[2.355s] [ext: omni.kit.viewport.window-106.0.8] startup
[2.371s] [ext: omni.kit.window.property-1.11.1] startup
[2.373s] [ext: omni.kit.window.content_browser-2.9.13] startup
[2.386s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[2.387s] [ext: omni.kit.viewport.utility-1.0.17] startup
[2.388s] [ext: omni.kit.property.usd-3.21.26] startup
[2.396s] [ext: omni.graph-1.135.0] startup
[2.502s] [ext: omni.physx-106.0.16] startup
[2.513s] [ext: omni.graph.ui-1.67.1] startup
[2.534s] [ext: omni.graph.action_core-1.1.4] startup
[2.541s] [ext: omni.graph.image.core-0.3.2] startup
[2.545s] [ext: omni.graph.action_nodes-1.21.3] startup
[2.553s] [ext: omni.graph.image.nodes-1.0.2] startup
[2.555s] [ext: omni.kit.numpy.common-0.1.2] startup
[2.557s] [ext: omni.graph.nodes-1.141.2] startup
[2.568s] [ext: omni.physics.tensors-106.0.16] startup
[2.579s] [ext: omni.physx.stageupdate-106.0.16] startup
[2.583s] [ext: omni.graph.action-1.101.1] startup
[2.584s] [ext: omni.warp.core-1.1.0] startup
Warp 1.1.0 initialized:
CUDA Toolkit 11.5, Driver 12.4
Devices:
"cpu" : "x86_64"
"cuda:0" : "NVIDIA GeForce RTX 3060 Laptop GPU" (6 GiB, sm_86, mempool enabled)
Kernel cache:
/root/.cache/warp/1.1.0
[2.701s] [ext: omni.isaac.dynamic_control-1.3.7] startup
[2.722s] [ext: omni.warp-1.1.0] startup
[2.758s] [ext: omni.kit.viewport.actions-105.2.8] startup
[2.777s] [ext: omni.kit.widget.imageview-1.0.3] startup
[2.780s] [ext: omni.debugdraw-0.1.3] startup
[2.785s] [ext: omni.physx.tensors-106.0.16] startup
[2.789s] [ext: omni.kit.window.cursor-1.1.2] startup
[2.791s] [ext: omni.isaac.version-1.1.0] startup
[2.792s] [ext: omni.kit.widget.toolbar-1.6.2] startup
[2.837s] [ext: omni.kit.manipulator.transform-104.7.5] startup
[2.857s] [ext: omni.kit.viewport.menubar.core-107.0.0] startup
[3.047s] [ext: omni.isaac.nucleus-0.1.2] startup
[3.049s] [ext: omni.kit.manipulator.tool.snap-1.4.5] startup
[3.054s] [ext: omni.kit.viewport.menubar.display-106.0.0] startup
[3.057s] [ext: omni.kit.window.quicksearch-2.4.4] startup
[3.062s] [ext: omni.physx.commands-106.0.16] startup
[3.067s] [ext: omni.usdphysics.ui-106.0.16] startup
[3.089s] [ext: omni.isaac.core-3.12.0] startup
[4.126s] [ext: omni.syntheticdata-0.6.7] startup
[4.152s] [ext: omni.kit.property.material-1.9.3] startup
[4.161s] [ext: omni.kit.manipulator.selector-1.1.1] startup
[4.164s] [ext: omni.physx.ui-106.0.16] startup
[4.199s] [ext: omni.fabric.commands-1.1.4] startup
[4.204s] [ext: omni.kit.manipulator.viewport-107.0.0] startup
[4.207s] [ext: omni.kit.viewport.manipulator.transform-106.0.0] startup
ViewportTransformManipulatorExt on_startup
[4.209s] [ext: omni.kit.property.physx-106.0.16] startup
[4.281s] [ext: omni.physx.demos-106.0.16] startup
[4.304s] [ext: omni.ui_query-1.1.2] startup
[4.308s] [ext: omni.kit.manipulator.prim.core-106.0.2] startup
[4.315s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[4.327s] [ext: omni.physx.vehicle-106.0.16] startup
[4.356s] [ext: omni.kit.ui_test-1.2.18] startup
[4.360s] [ext: omni.kit.manipulator.prim.fabric-106.0.1] startup
[4.362s] [ext: omni.kit.manipulator.prim.usd-106.0.1] startup
[4.364s] [ext: omni.physx.camera-106.0.16] startup
[4.376s] [ext: omni.physx.cct-106.0.16] startup
[4.387s] [ext: omni.kit.manipulator.selection-104.0.9] startup
[4.395s] [ext: omni.kit.manipulator.prim-106.0.0] startup
[4.396s] [ext: omni.kit.window.toolbar-1.6.1] startup
[4.403s] [ext: omni.kit.widget.layers-1.7.9] startup
[4.423s] [ext: omni.kit.graph.delegate.default-1.2.2] startup
[4.426s] [ext: omni.physx.graph-106.0.16] startup
[4.441s] [ext: omni.graph.ui_nodes-1.24.1] startup
[4.447s] [ext: omni.physx.supportui-106.0.16] startup
[4.474s] [ext: omni.kit.graph.usd.commands-1.3.1] startup
[4.478s] [ext: omni.kit.graph.editor.core-1.5.3] startup
[4.483s] [ext: omni.physx.telemetry-106.0.16] startup
[4.490s] [ext: omni.graph.bundle.action-2.0.4] startup
[4.490s] [ext: omni.command.usd-1.0.3] startup
[4.491s] [ext: omni.kit.widget.material_preview-1.0.16] startup
[4.495s] [ext: omni.isaac.debug_draw-0.7.1] startup
[4.502s] [ext: omni.kit.window.material_graph-1.8.15] startup
[4.553s] [ext: omni.sensors.tiled-0.0.3] startup
[4.556s] [ext: omni.physx.bundle-106.0.16] startup
[4.556s] [ext: omni.graph.scriptnode-1.18.2] startup
[4.558s] [ext: omni.isaac.ui-0.15.1] startup
[4.573s] [ext: omni.replicator.core-1.11.8] startup
2024-07-04 13:38:40 [4,582ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[4.696s] [ext: omni.sensors.nv.common-1.0.1] startup
[4.708s] [ext: omni.isaac.range_sensor-3.0.1] startup
[4.745s] [ext: omni.isaac.core_nodes-1.15.4] startup
[4.770s] [ext: omni.sensors.nv.wpm-1.0.0] startup
[4.773s] [ext: omni.sensors.nv.materials-1.0.0] startup
[4.776s] [ext: omni.isaac.lula-3.0.1] startup
[4.787s] [ext: omni.isaac.surface_gripper-1.0.1] startup
[4.796s] [ext: omni.isaac.motion_generation-7.0.1] startup
[4.839s] [ext: omni.isaac.manipulators-2.1.0] startup
[4.851s] [ext: omni.sensors.nv.radar-1.0.1] startup
[4.859s] [ext: omni.sensors.nv.lidar-1.0.1] startup
[4.865s] [ext: omni.isaac.franka-0.4.1] startup
[4.869s] [ext: omni.isaac.dofbot-0.3.2] startup
[4.881s] [ext: omni.isaac.sensor-11.3.0] startup
[4.989s] [ext: omni.isaac.cortex-0.3.8] startup
[4.992s] [ext: omni.isaac.universal_robots-0.3.5] startup
[4.999s] [ext: omni.isaac.quadruped-1.4.5] startup
[5.040s] [ext: omni.isaac.cortex.sample_behaviors-1.0.5] startup
[5.044s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[5.055s] [ext: omni.kit.graph.delegate.modern-1.10.6] startup
[5.058s] [ext: omni.isaac.wheeled_robots-2.1.3] startup
[5.061s] [ext: omni.graph.window.core-1.107.1] startup
[5.080s] [ext: omni.isaac.examples-2.0.0] startup
[5.213s] [ext: omni.importer.mjcf-1.1.0] startup
[5.246s] [ext: omni.graph.window.generic-1.23.1] startup
[5.259s] [ext: omni.importer.onshape-0.7.1] startup
[5.270s] [ext: omni.isaac.manipulators.ui-1.2.1] startup
[5.273s] [ext: omni.importer.urdf-1.14.1] startup
[5.324s] [ext: omni.kit.streamsdk.plugins-3.2.1] startup
[5.324s] [ext: omni.isaac.robot_description_editor-2.4.0] startup
[5.347s] [ext: omni.isaac.examples_nodes-0.1.3] startup
[5.349s] [ext: omni.isaac.cloner-0.8.1] startup
[5.355s] [ext: omni.isaac.extension_templates-1.7.0] startup
[5.359s] [ext: omni.isaac.gain_tuner-1.1.1] startup
[5.368s] [ext: omni.isaac.gym-0.11.3] startup
[5.371s] [ext: omni.isaac.import_wizard-0.5.2] startup
[5.378s] [ext: omni.isaac.kit-1.10.0] startup
[5.387s] [ext: omni.kit.mesh.raycast-105.4.0] startup
[5.393s] [ext: omni.anim.curve.core-1.1.13] startup
[5.409s] [ext: omni.isaac.block_world-1.0.0] startup
[5.413s] [ext: omni.anim.shared.core-106.0.0] startup
[5.422s] [ext: omni.anim.curve.ui-1.3.16] startup
[5.432s] [ext: omni.isaac.occupancy_map-1.0.1] startup
[5.442s] [ext: omni.isaac.merge_mesh-1.0.1] startup
[5.450s] [ext: omni.kit.widget.stage_icons-1.0.3] startup
[5.461s] [ext: omni.anim.curve.bundle-1.2.3] startup
[5.462s] [ext: omni.isaac.occupancy_map.ui-1.1.0] startup
[5.470s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[5.476s] [ext: omni.kit.window.stage-2.5.9] startup
[5.481s] [ext: omni.kit.usd.collect-2.2.21] startup
[5.487s] [ext: omni.kit.browser.core-2.3.11] startup
[5.496s] [ext: omni.kit.menu.stage-1.2.5] startup
[5.498s] [ext: omni.kit.tool.collect-2.2.12] startup
[5.505s] [ext: omni.kit.browser.folder.core-1.9.12] startup
[5.512s] [ext: omni.kit.search.files-1.0.4] startup
[5.517s] [ext: omni.isaac.physics_inspector-0.1.4] startup
[5.520s] [ext: omni.isaac.asset_browser-0.4.6] startup
[5.527s] [ext: omni.kit.usda_edit-1.1.11] startup
[5.539s] [ext: omni.scene.optimizer.core-106.0.4] startup
[5.553s] [ext: omni.isaac.scene_blox-0.1.1] startup
[5.556s] [ext: omni.isaac.surface_gripper.ui-1.0.0] startup
[5.558s] [ext: omni.isaac.synthetic_recorder-1.7.3] startup
[5.566s] [ext: omni.kit.converter.common-201.0.2] startup
[5.571s] [ext: omni.isaac.throttling-1.1.0] startup
[5.574s] [ext: omni.isaac.utils-1.0.1] startup
[5.578s] [ext: omni.isaac.wheeled_robots.ui-1.1.1] startup
[5.584s] [ext: omni.kit.converter.cad_core-201.0.2] startup
[5.596s] [ext: omni.kit.converter.jt_core-201.0.2] startup
[5.614s] [ext: omni.kit.tool.asset_importer-2.5.5] startup
[5.623s] [ext: omni.kit.actions.window-1.1.1] startup
[5.630s] [ext: omni.isaac.window.about-1.1.0] startup
[5.634s] [ext: omni.kit.browser.asset-1.3.9] startup
[5.638s] [ext: omni.kit.selection-0.1.4] startup
[5.642s] [ext: omni.kit.converter.cad-201.0.2] startup
[5.648s] [ext: omni.kit.hotkeys.window-1.4.5] startup
[5.658s] [ext: omni.kit.manipulator.camera-105.0.5] startup
[5.664s] [ext: omni.kit.menu.aov-1.1.3] startup
[5.666s] [ext: omni.kit.menu.edit-1.1.19] startup
[5.673s] [ext: omni.kit.property.audio-1.0.11] startup
[5.676s] [ext: omni.hydra.scene_api-0.1.2] startup
[5.683s] [ext: omni.kit.property.camera-1.0.6] startup
[5.685s] [ext: omni.kit.property.geometry-1.3.0] startup
[5.692s] [ext: omni.kit.property.light-1.0.8] startup
[5.695s] [ext: omni.kit.property.render-1.1.1] startup
[5.698s] [ext: omni.kit.property.transform-1.5.1] startup
[5.704s] [ext: omni.kit.menu.file-1.1.9] startup
[5.709s] [ext: omni.kit.core.collection-0.1.7] startup
[5.711s] [ext: omni.kit.preferences.animation-1.1.7] startup
[5.716s] [ext: omni.isaac.menu-0.4.1] startup
[5.721s] [ext: omni.kit.property.bundle-1.2.11] startup
[5.727s] [ext: omni.kit.menu.create-1.0.12] startup
[5.732s] [ext: omni.isaac.physics_utilities-0.1.4] startup
[5.739s] [ext: omni.kit.property.collection-0.1.17] startup
[5.744s] [ext: omni.kit.property.isaac-0.2.3] startup
[5.748s] [ext: omni.mdl.usd_converter-1.0.18] startup
[5.751s] [ext: omni.kit.property.layer-1.1.5] startup
[5.757s] [ext: omni.kit.stagerecorder.core-105.0.5] startup
[5.762s] [ext: omni.kit.scripting-106.0.1] startup
[5.773s] [ext: semantics.schema.property-1.0.2] startup
[5.776s] [ext: omni.kit.stage.copypaste-1.2.2] startup
[5.780s] [ext: omni.kit.stage.mdl_converter-1.0.5] startup
[5.783s] [ext: omni.kit.widgets.custom-1.0.8] startup
[5.796s] [ext: omni.kit.stagerecorder.ui-105.0.6] startup
[5.802s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[5.806s] [ext: omni.kit.stage_column.payload-2.0.0] startup
[5.810s] [ext: omni.isaac.range_sensor.examples-1.0.0] startup
[5.833s] [ext: omni.graph.window.action-1.25.2] startup
[5.850s] [ext: omni.kit.variant.editor-106.0.0] startup
[5.868s] [ext: omni.kit.stagerecorder.bundle-105.0.2] startup
[5.869s] [ext: omni.kit.thumbnails.usd-1.0.9] startup
[5.949s] [ext: omni.isaac.range_sensor.ui-1.0.0] startup
[5.953s] [ext: omni.kit.variant.presenter-105.1.2] startup
[5.962s] [ext: omni.kit.viewport.menubar.camera-105.1.8] startup
[5.973s] [ext: omni.kit.viewport.menubar.lighting-106.0.2] startup
[5.989s] [ext: omni.rtx.window.settings-0.6.15] startup
[6.003s] [ext: omni.kit.viewport.menubar.render-106.1.2] startup
[6.011s] [ext: omni.kit.widget.live-2.1.6] startup
[6.015s] [ext: omni.kit.widget.calendar-1.0.8] startup
[6.019s] [ext: omni.kit.viewport.menubar.settings-106.0.0] startup
[6.029s] [ext: omni.kit.widget.collection-0.1.18] startup
[6.038s] [ext: omni.kit.viewport.rtx-104.0.1] startup
[6.039s] [ext: omni.kit.viewport_widgets_manager-1.0.8] startup
[6.045s] [ext: omni.kit.widget.cache_indicator-2.0.8] startup
[6.238s] [ext: omni.kit.widget.extended_searchfield-1.0.27] startup
2024-07-04 13:38:42 [6,237ms] [Warning] [omni.kit.widget.cache_indicator.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
[6.647s] [ext: omni.kit.widget.timeline-105.0.1] startup
[6.652s] [ext: omni.kit.window.collection-0.1.22] startup
[6.655s] [ext: omni.kit.window.commands-0.2.5] startup
[6.656s] [ext: omni.kit.window.console-0.2.12] startup
[6.666s] [ext: omni.kit.window.imageviewer-1.0.7] startup
[6.672s] [ext: omni.kit.window.privacy-0.1.0] startup
[6.673s] [ext: omni.kit.tool.asset_exporter-1.3.3] startup
[6.676s] [ext: omni.kit.window.script_editor-1.7.6] startup
[6.678s] [ext: omni.kit.window.status_bar-0.1.6] startup
[6.683s] [ext: omni.kit.window.usd_paths-1.0.6] startup
[6.686s] [ext: omni.usd.metrics.assembler-106.0.1] startup
[6.693s] [ext: omni.replicator.isaac-1.15.0] startup
[6.708s] [ext: omni.kit.browser.material-1.5.2] startup
[6.715s] [ext: omni.replicator.replicator_yaml-2.0.5] startup
[6.725s] [ext: omni.kit.ui.actions-1.0.1] startup
[6.730s] [ext: omni.rtx.settings.core-0.6.0] startup
[6.737s] [ext: omni.simready.explorer-1.0.26] startup
[6.754s] [ext: omni.usd.metrics.assembler.physics-106.0.2] startup
[6.758s] [ext: omni.kit.tool.remove_unused.core-0.1.2] startup
[6.761s] [ext: omni.kit.window.material-1.5.7] startup
[6.769s] [ext: omni.usd.metrics.assembler.ui-106.0.2] startup
[6.774s] [ext: omni.kit.menu.common-1.1.5] startup
[6.776s] [ext: omni.kit.renderer.cuda_interop-1.0.1] startup
[6.778s] [ext: omni.kit.quicklayout-1.0.6] startup
[6.779s] [ext: omni.kit.window.title-1.1.3] startup
[6.781s] [ext: semantics.schema.editor-0.3.4] startup
[6.784s] [ext: omni.kit.tool.remove_unused.controller-0.1.3] startup
[6.785s] [ext: omni.isaac.robot_assembler-1.5.0] startup
[6.790s] [ext: omni.isaac.camera_inspector-0.2.1] startup
[6.791s] [ext: omni.kit.livestream.core-3.2.0] startup
[6.796s] [ext: omni.kit.window.stats-0.1.6] startup
[6.798s] [ext: omni.isaac.app.setup-0.13.2] startup
[6.800s] Isaac Sim Headless Native Version: 4.0.0-rc.21
[6.807s] [ext: omni.isaac.assets_check-0.3.4] startup
[6.809s] [ext: omni.kit.profiler.window-2.2.1] startup
2024-07-04 13:38:42 [6,791ms] [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
[6.817s] [ext: omni.isaac.sim.base-4.0.0] startup
[6.819s] [ext: omni.kit.livestream.native-4.1.0] startup
Failed to open [/var/run/utmp]
Active user not found. Using default user [kiosk]Streaming server started.
[12.254s] [ext: omni.isaac.sim.headless.native-4.0.0] startup
2024-07-04 13:38:51 [15,596ms] [Warning] [omni.fabric.plugin] No source has valid data array=0x22db80e0 usdValid=0 cpuValid=0
2024-07-04 13:38:51 [15,823ms] [Warning] [omni.fabric.plugin] No source has valid data array=0x22fbee40 usdValid=0 cpuValid=0
It hangs here indefinitely.