Unable to reliably produce lidar data in a headless simulation

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: A40-48Q
  • Driver Version: 550.127.05

Topic Description

Detailed Description

I am trying to have a scripted headless simulation with two sensors, a camera and a lidar. I need to access the rgb data and the lidar points in the script. I am using an rgb annotator and an RtxSensorCpuIsaacCreateRTXLidarScanBuffer annotator for this purpose. The script uses the replicator orchestrator in order to step through the simulation 10 times. I expected that with every step I can use get_data() on the annotators to retrieve the data. Unfortunately, the lidar sensor only produces data for step 1, while for the other steps the data is empty. The rgb data can be retrieved as expected for every simulation step.

Steps to Reproduce

  1. Run the attached script test06.py using Isaac Sim’s python.sh while having the usd file test12flat.usd in the same folder

Error Messages

[7.107s] app ready                                                                      
2025-07-02 08:49:19 [7,209ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit buffer size 7374781440                                                                 
2025-07-02 08:49:19 [7,210ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : valid false, within: false
2025-07-02 08:49:19 [7,210ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : decrement: 167690, decrement size: 7301033856
2025-07-02 08:49:19 [7,210ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : New limit 9748724 (slope: 439, intercept: 13179904)
2025-07-02 08:49:19 [7,210ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit buffer size 4287352704
2025-07-02 08:49:19 [7,210ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : valid true, within: true
2025-07-02 08:49:19 [7,358ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults-transient/meshlights/forceDisable'
2025-07-02 08:49:19 [7,416ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults/post/dlss/execMode'
[10.765s] Simulation App Startup Complete                                               
Step 0                                                                                  
2025-07-02 08:49:23 [11,415ms] [Warning] [omni.syntheticdata.plugin] OgnSdPostRenderVarToHost : rendervar copy from texture directly to host buffer is counter-performant. Pleas
e use copy from texture to device buffer first.                                                                                                                                 
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                           
Step 1                         
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 1209                                                                                                                                                                
Step 2                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 3                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 4                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 5                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 6                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 7                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0                                                                                                                                                                   
Step 8                                                                                                                                                                          
RGB Data: 15769600                                                                                                                                                              
Lidar Data: 0
Step 9
RGB Data: 15769600
Lidar Data: 0
2025-07-02 08:49:25 [13,627ms] [Warning] [carb] Plugin interface for a client: omni.hydratexture.plugin was already released.
2025-07-02 08:49:25 [13,627ms] [Warning] [carb] Plugin interface for a client: omni.hydratexture.plugin was already released.
[13.781s] Simulation App Shutting Down

As you can see, the length of the data the lidar produces is 1209 for step 1, and 0 for all other steps.

test06.zip (577.6 KB)

Hi @bruno.vetter. Thanks for your files. I increased the number of steps and I can see more steps with values from lidar. I wonder if it is related to lidar data frequency. Let me reach out to the internal team about your issue.

Just curious but why do you have to run in headless? Does it work if you run it normally?

We are running a large number of simulations as batches orchestrated by a ci pipeline, so there is no UI present. That’s why we want to run headless. If I run it localy with headless=False, I get the following output:

[10.159s] Simulation App Startup Complete
Step 0
2025-07-04 06:08:25 [10,870ms] [Warning] [omni.syntheticdata.plugin] OgnSdPostRenderVarToHost : rendervar copy from texture directly to host buffer is counter-performant. Please use copy from texture to device buffer first.
RGB Data: 15769600
Lidar Data: 0
Step 1
RGB Data: 15769600
Lidar Data: 1209
Step 2
RGB Data: 15769600
Lidar Data: 0
Step 3
RGB Data: 15769600
Lidar Data: 0
Step 4
RGB Data: 15769600
Lidar Data: 0
Step 5
RGB Data: 15769600
Lidar Data: 1209
Step 6
RGB Data: 15769600
Lidar Data: 0
Step 7
RGB Data: 15769600
Lidar Data: 0
Step 8
RGB Data: 15769600
Lidar Data: 1209
Step 9
RGB Data: 15769600
Lidar Data: 0
2025-07-04 06:08:27 [12,961ms] [Warning] [carb] Plugin interface for a client: omni.hydratexture.plugin was already released.
2025-07-04 06:08:27 [12,961ms] [Warning] [carb] Plugin interface for a client: omni.hydratexture.plugin was already released.
[13.087s] Simulation App Shutting Down```