4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Operating System
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):
GPU Information
Model: 4x A6000
Driver Version: 560.35.03
Topic Description
Detailed Description
Isaac Sim hangs at startup when I run a standalone headless script through SSH.
Steps to Reproduce
ssh <username>@<remote_machine_ip>
cd <isaacsim_root>
./python.sh ./standalone_examples/tutorials/getting_started.py
The getting_started.py is modified so that it runs in headless mode.
Error Messages
No error in the terminal.
In the log, the process is stuck at
...
2025-05-06 20:42:32 [158,977ms] [Info] [gpu.foundation.plugin] *** Waiting for RtPso async group async compilation: 135 seconds so far
2025-05-06 20:42:37 [163,978ms] [Info] [gpu.foundation.plugin] *** Waiting for RtPso async group async compilation: 140 seconds so far
Hi there and thanks for your question. You cannot run anything KIT in SSH. Even to run headless we require you to still be logged in through a vGPU interface. Kit is GPU accelerated. It has to be run in a gpu accelerated viewer. Headless or not.
Thanks for the reply! In that case, what is the best way to run a standalone Python script on a remote headless server? I couldn’t find any relevant information, as the documentation mainly focuses on running the remote streaming client.
You want to run a standalone python script, but talking to what? Kit? If it is standalone, how are you connecting it to Omniverse? It is not so much about a headless “server”. More about running Isaac Sim headless. The server needs to be fully GPU accelerated and available. It is the KIT session that is run headless. Not the server. To connect to the server you use normal remote viewing software. Then you can use normal windows CMD to run scripts to a headless session.
By “standalone” I meant the Standalone Python workflow described here Workflows — Isaac Sim Documentation
where I have a Python script that starts with
from isaacsim import SimulationApp
simulation_app = SimulationApp({"headless": True})
By headless server, I meant a server with no desktop environment installed, but it is still fully GPU accelerated and available. Because there is no desktop environment, remote viewing software would not work.
Yes exactly. And that is the issue. You cannot run it on a headless server. You can run Isaac Sim headless but it is only an option. The default is to run it fully with GUI, and even though you flag it for headless it still requires a vGPU setup. So no headless server and no SSH. Install a normal Ubuntu GUI and then you can vGPU in and run your headless isaac command as normal.