GPU crash

Isaac Sim Version

4.5.0

Operating System

Ubuntu 22.04

GPU Information

  • Model: RTX A6000
  • Driver Version: 570.86.10

Topic Description

I get a GPU crash from the isaac-tiamat/GroundControl repo, which Nvidia engineers have access to.

Detailed Description

Steps to Reproduce

  1. Follow the setup instructions at Sign in · GitLab
  2. python GroundControl/scripts/environments/teleoperation/teleop_se2_agent.py --task Isaac-Navigation-Flat-Spot-Play-v0 --num_envs 1 --teleop_device keyboard

Error Messages

IsaacSim is stuck loading at 50%. The console output contains errors as follows:

2025-04-28 20:36:35 [52,606ms] [Warning] [rtx.rtxsensor.plugin] Using User-specified coordinate frame quaternion (0.000000, 0.000000, 0.000000, 1.000000) in RTXSensor

Warning: Possible version incompatibility. Attempting to load omni::fabric::IPrimBucketList with version v0.4 against v0.3.

2025-04-28 20:36:36 [52,930ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault occured on virtual address(0x0000004d00000000)

2025-04-28 20:36:36 [52,930ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault no associated data found closest allocations were:

2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] 1: AddressRange(0x0000000000000000 - 0x0000000000000000) with name(dummy to ignore)

2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] 2: AddressRange(0xffffffffffffffff - 0xffffffffffffffff) with name(dummy to ignore)

2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0

2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0

2025-04-28 20:36:36 [52,931ms] [Error] [carb.graphics-vulkan.plugin] GPU crash is detected. Trying to write crash dump into: /homes/elenius/miniconda3/envs/tiamat/lib/python3.10/site-packages/omni/logs/Kit/Isaac-Sim/4.5/kit_20250428_133543-0.nv-gpudmp

2025-04-28 20:36:36 [52,931ms] [Error] [carb.graphics-vulkan.plugin] GPU crash dump is successfully written

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault occured on virtual address(0x0000004d00000000)

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault no associated data found closest allocations were:

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] 1: AddressRange(0x0000000000000000 - 0x0000000000000000) with name(dummy to ignore)

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] 2: AddressRange(0xffffffffffffffff - 0xffffffffffffffff) with name(dummy to ignore)

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0

2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0

2025-04-28 20:36:36 [52,936ms] [Error] [carb.graphics-vulkan.plugin] GPU crash is detected. Trying to write crash dump into: /homes/elenius/miniconda3/envs/tiamat/lib/python3.10/site-packages/omni/logs/Kit/Isaac-Sim/4.5/kit_20250428_133543-1.nv-gpudmp

2025-04-28 20:36:36 [52,936ms] [Error] [carb.graphics-vulkan.plugin] GPU crash dump is successfully written

zip file with the two .nv-gpudmp files mentioned above attached.

gpudmp.zip (18.0 KB)

We don’t have access to the GitLab site. Is this IsaacLab related?