Omni.datastore error

Hello,

I’m running isaac sim with python standalone.

The error related to DerivedDataCache is keep showing up, which I could not find the root cause.

2026-03-26T17:32:58Z [651ms] [Error] [omni.datastore] Failed to acquire exclusive lock to data store (256>=256)
2026-03-26T17:32:58Z [651ms] [Error] [omni.datastore] Failed to create local file data store at '/home/kzl/Projects/isaacsim/kit/cache/DerivedDataCache'

This is the command I’m running from headless server:

`./python.sh my_folder/execute.py --headless “$@”`

# In executor.py, I'm running:    
from isaacsim import SimulationApp
simulation_app = SimulationApp({
        "headless":          args.headless,
        "limit_cpu_threads": 16,
    })

This is a file-lock contention issue in omni.datastore’s DerivedDataCache (used by PhysX collision cooking). The (256>=256) means 256 concurrent processes already hold locks on the shared cache directory.

Same root cause as this earlier thread — the limit was raised from 64 to 256 in newer versions, but you’re exceeding even the new limit.

Workarounds:

  1. Disable UJITSO PhysX cooking (eliminates the cache that causes lock contention):
                                                                                                                                                                                                                                                                                                                            
   ./python.sh my_folder/execute.py --headless --/physics/cooking/ujitsoCollisionCooking=false "$@"                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                            
  1. Isolate cache directories per process so they don’t share locks:
                                                                                                                                                                                                                                                                                                                            
   export OMNI_LOCALCACHE_PATH="/tmp/isaacsim_cache_$$"                                                                                                                                                                                                                                                                     
   ./python.sh my_folder/execute.py --headless "$@"                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                            
  1. If neither helps, could you share:
  • How many concurrent processes you’re running?
  • Whether this is on a Slurm cluster or a single machine?
  • Your GPU model and driver version?

This will help us determine whether the lock limit should be made configurable or removed entirely for headless/batch workloads.