[Fatal] [rtx.scenedb.plugin] Waiting on Semaphore 1948 for longer than 60s:

Omniverse Isaac Sim
,
1

Isaac Sim Version

4.5.0

Operating System

Ubuntu 22.04

GPU Information

  • Model: Nvidia L4
  • Driver Version: 535.129.03

Semaphore error

Detailed Description

I am trying to generate synthetic data with complex scene. sample synthetic image is provided below

The scene has multiple containers and each containers has randomly 1-50 samples. All samples are individually placed randomly for one scene. and All sample usd prims are deleted and loaded again for each scene.

During the image generation there is no vram issues has out of 24 gb of vram only 11-12gb is used by isaac sim application

# nvidia-smi output during image rendering process

Fri May 30 11:06:15 2025       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA L4                      Off | 00000000:00:03.0 Off |                    0 |
| N/A   76C    P0              59W /  72W |  11793MiB / 23034MiB |    100%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      3946    C+G   /isaac-sim/kit/python/bin/python3         11749MiB |
+---------------------------------------------------------------------------------------+
  • I am getting error as hown below station semaphore timeout, im not gpu expert so not able to solve this.
  • The images are rendered and annotations are stored properly.
  • But I cannot generate all the images, like if i put code to generate 2000 images the error semaphore timeout continues all the time and after some images like 370-400 the process stops with error device not found.
  • I guess it caused due to this semaphore error where threads are blocking.
### Error Messages
2025-05-30 11:05:50 [2,729,383ms] [Error] [rtx.scenedb.plugin] Failed to begin copy command list (error code: 1)
2025-05-30 11:05:50 [2,729,465ms] [Fatal] [rtx.scenedb.plugin] Waiting on Semaphore 1948 for longer than 60s: Failure to complete CopyCommandList: Copy Context Geometry copy engine command list command list
2025-05-30 11:05:50 [2,729,628ms] [Error] [rtx.scenedb.plugin] Failed to begin copy command list (error code: 1)
2025-05-30 11:05:50 [2,729,710ms] [Fatal] [rtx.scenedb.plugin] Waiting on Semaphore 1948 for longer than 60s: Failure to complete CopyCommandList: Copy Context Geometry copy engine command list command list

Screenshots or Videos

image
image1920×1105 386 KB

3

Could you provide the full log file? This will help us check if any other errors occur before the issue starts.

4

I have uploaded the full log file here. This log file has same error but it was generated on local pc. In cloud vm I used docker and could not find log file inside docker container.
kit_20250528_152315.log (55.3 MB)

5

Could you share a minimal reproduction sample? This will allow us to replicate the problem efficiently and identify the root cause.

6

I could not reproduce that error locally with other code. Is it okay if can share all code files and assets require to run the program. (maybe not here in public forum but as private message something?)

7

Please share them via a private message. Thanks.