Kit App Crashes with Headless SSH Linux Config (GLFW Errors)

brainstencil · February 20, 2025, 6:30pm

I’m having difficulty getting my headless setup to work with dual RTX 4090 GPUs on Ubuntu 22.04 LTS using the NVIDIA driver. I’ve installed the official driver (version 535.129.03) via the .run installer (installed after cleaning up previous drivers), and I’m trying to run X in a headless mode with a virtual display for use with Omniverse.

System Details:
OS: Ubuntu 22.04 LTS
GPUs: 2 × NVIDIA GeForce RTX 4090
Driver Version: 535.129.03 (installed using the official .run file)
Connection: Headless (no physical monitor)

I assume there is something wrong with the X server, I can’t launch the default composer template:

[0.601s] [ext: carb.windowing.plugins-1.0.0] startup 
2025-02-17 05:51:27 [561ms] [Warning] [carb.windowing-glfw.plugin] GLFW initialization failed. 
2025-02-17 05:51:27 [561ms] [Warning] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.5]) (impl: carb.windowing-glfw.plugin) [0.603s] [ext: omni.kit.renderer.init-0.0.0] startup 
2025-02-17 05:51:27 [620ms] [Warning] [omni.platforminfo.plugin] failed to open the default display. Can't verify X Server version.

±--------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 15609 G /usr/lib/xorg/Xorg 14MiB |
| 1 N/A N/A 15609 G /usr/lib/xorg/Xorg 14MiB |
±--------------------------------------------------------------------------------------+

xorg.conf:
xorg.conf.txt (2.4 KB)

Richard3D · February 23, 2025, 2:11pm

A physical display must be connected to your GPU, unless you are running kit/Create headless with --no-window

brainstencil · February 24, 2025, 5:59pm

Hi @Richard3D thanks for your help.

I managed to get past the GLFW errors, but still can’t launch the kit app. First, I had CUDA errors, so I installed them. But that doesn’t help.

used nvidia-xconfig to generate an xorg conf
Started an xorg server on Display=:1
Tested that GLFW works
installed cuda
Can’t run omniverse because the physx Python package fails to install due to inability to find cuda libs

I’m running nucleus on the same server that I’m trying to get the kit app to load on this same server. Is that a problem?

Also, I’m not sure the --no-window command is working, how do I do that properly?

.\repo.sh launch -d --no-window

Richard3D · February 25, 2025, 3:34am

Yes “.\repo.sh launch -d --no-window” should work fine. Cuda? You should not need Cuda. I don’t understand. Does this run normally with a monitor, without running headless? Start back at the beginning. Does it run at all, normally on this desktop with a monitor? If it does, then it should run headless. It does require anything else special.

brainstencil · February 26, 2025, 12:09am

I’m just trying to launch the default composer app from the kit app template on Linux.

So… Yes it does work on other workstations, but not on this one. It is a rented cloud GPU workstation that does not have a monitor (and it can’t have one, so I can’t test).

I was surprised that I was seeing CUDA errors as well. First, can you tell me what I’m doing wrong to get --no-window working?

$ ./repo.sh launch -d --no-window

error: unrecognized arguments: --no-window
usage: repo launch [-h] [--container] [-d] [-p FROM_PACKAGE] [-n APP_NAME]
                   {my_company.my_usd_composer_ovc.kit,my_company.my_usd_composer.kit,my_company.my_usd_composer_streaming.kit,my_company.my_usd_composer2.kit,my_company.my_usd_composer2_streaming.kit,my_company.my_usd_composer2_ovc.kit}
                   ...

Richard3D · February 26, 2025, 2:16am

I’m going to check, but I think you add the no window command directly to the actual kit file. It’s an argument that goes there and not on the repo command.

Richard3D · February 26, 2025, 2:48pm

With repo launch you need to have the separator – everything before it will be a repo launch argument everything after it will be forwarded to kit, so the correct launch argument with no window would be repo.sh launch -d – --no-window

brainstencil · February 27, 2025, 10:18pm

Hi,

I’m still seeing cuda errors in the startup carb.logs , even though I was able to get no-window to work.

The syntax is actually 2 dashes after the ‘-d’:

./repo.sh launch -d -- --no-window

I’m still seeing these libcuda issues and a crash:

Here:

[8.503s] [ext: omni.kit.livestream.webrtc-6.0.0] startup
Warning: Possible version incompatibility. Attempting to load carb::cudainterop::CudaInterop with version v0.15 against v0.14.

and here:

[11.804s] [ext: my_company.my_usd_composer_streaming-0.1.0] startup
[12.120s] app ready
2025-02-27 22:15:37 [12,435ms] [Error] [omni.physx.plugin] PhysX error: Could not load libcuda.so: libcuda.so: cannot open shared object file: No such file or directory
, FILE /builds/omniverse/physics/physx/source/physx/src/gpu/PxPhysXGpuModuleLoader.cpp, LINE 204
2025-02-27 22:15:37 [12,435ms] [Warning] [omni.kvdb.plugin] Disabling key-value database because another kit process is locking it
2025-02-27 22:15:37 [12,440ms] [Warning] [carb.audio.device] audio device is misconfigured or broken {deviceIndex = 0, name = 'default'} (The device is likely misconfigured, check your $HOME/.asoundrc)
2025-02-27 22:15:37 [12,440ms] [Warning] [carb.audio.output] failed to retrieve the capabilities for device 0 {result = eDeviceLost (2)}
2025-02-27 22:15:37 [12,440ms] [Warning] [carb.audio.context] failed to set the requested output during context creation.  Using a null streamer instead {result = eDeviceLost (2)}
2025-02-27 22:15:38 [12,496ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults/ecoMode/enabled'
2025-02-27 22:15:38 [12,496ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults/post/aa/op'
2025-02-27 22:15:38 [12,497ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults/sceneDb/ambientLightIntensity'
2025-02-27 22:15:38 [12,497ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults/verifyDriverVersion/enabled'
2025-02-27 22:15:38 [12,604ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit buffer size 8274665856
2025-02-27 22:15:38 [12,604ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : valid false, within: false
2025-02-27 22:15:38 [12,604ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : decrement: 167690, decrement size: 8191919488
2025-02-27 22:15:38 [12,605ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : New limit 8692359 (slope: 493, intercept: 7537024)
2025-02-27 22:15:38 [12,605ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit buffer size 4289241344
2025-02-27 22:15:38 [12,605ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : valid true, within: true
2025-02-27 22:15:38 [12,605ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit buffer size 4289241344
2025-02-27 22:15:38 [12,605ms] [Warning] [rtx.scenedb.plugin] SceneDbContext : TLAS limit : valid true, within: true
2025-02-27 22:15:38 [12,934ms] [Warning] [omni.usd-abi.plugin] No setting was found for '/rtx-defaults-transient/meshlights/forceDisable'
2025-02-27 22:15:39 [0ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred.  If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds.  This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2025-02-27 22:15:40 [105ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] compressing 3 files into '/home/ops/.local/share/ov/data/Kit/My USD Composer Streaming/0.1/0d7337b1-c366-4dbf-a3b719a6-d5bec359.dmp.zip' for upload.

here’s the log file: kit_20250227_221525.log (866.7 KB)

Any ideas? Thanks!

brainstencil · February 27, 2025, 11:16pm

I decided to try to install CUDA again, and after installing, I ran:

sudo ln -s /lib/x86_64-linux-gnu/libcuda.so.1 /lib/x86_64-linux-gnu/libcuda.so

Now I am able to get the app to where it says

RTX Ready

Was that a bad idea? I never saw the README mention needing to install CUDA myself.

I still seeing a lot of warnings, and the render related warnings concern me:

[0.620s] [ext: omni.kit.renderer.init-0.0.0] startup

|---------------------------------------------------------------------------------------------|
| Driver Version: 535.183.01    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 4090          | Yes: 0 |     | 23274   MB | 10de      | 0          |
|     |                                  |        |     |            | 2684      | 1be1fa63.. |
|     |                                  |        |     |            | 2         |            |
|---------------------------------------------------------------------------------------------|
| 1   | NVIDIA GeForce RTX 4090          | Yes: 1 |     | 23274   MB | 10de      | 0          |
|     |                                  |        |     |            | 2684      | 38370699.. |
|     |                                  |        |     |            | 82        |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 5.15.0-88-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101004 (1.21.1.4)
| Processor: Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
| Cores: 36 | Logical Cores: 72
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 515836 | Free Memory: 508779
| Total Page/Swap (MB): 0 | Free Page/Swap: 0
|---------------------------------------------------------------------------------------------|
2025-02-27 22:37:03 [1,754ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 0
2025-02-27 22:37:03 [1,754ms] [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 1
...
[5.345s] [ext: omni.hydra.rtx-1.0.0] startup
2025-02-27 22:37:07 [5,305ms] [Warning] [omni.log] Source: omni.hydra was already registered.
[5.373s] [ext: omni.hydra.engine.stats-1.0.3] startup

Are any of these warnings important to address? If so can you point me in the right direction?

Richard3D · February 28, 2025, 12:21am

If you have RTX READY then you are good.

brainstencil · February 28, 2025, 1:07am

Thanks!

system · March 14, 2025, 1:08am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Kit or Audio2Face Apps cannot start after enabled viewport extension Audio2Face (closed)	3	483	October 12, 2022
Omniverse kit Start Failure General Discussion	12	3045	January 15, 2021
Ubuntu 20.04 crash with Create Beta 2021.3.1 Driver 460.67 Samples & Examples	19	1369	September 25, 2023
X.org crashes on Ubuntu 17.10 with driver nvidia-384 after upgrade Linux	27	16998	January 31, 2018
Arbitrary Crashes / Segfaults with RTX 3070 on current driver-455 on Ubuntu 20.04 kernel 5.4.0-58-generic Linux	23	2159	February 25, 2021
Omniverse Kit Remote Client show python script visuals Core Platform python , rl	10	3902	October 12, 2021
Frequent catastrophic crashes on a multiple GPU machine CUDA Setup and Installation	8	4676	October 22, 2017
OpenGL, NVIDIA and Ubuntu 14.04 issues Linux	28	17362	September 22, 2017
NVidia driver 520.61.05 / Cuda 11.8 / RTX 3090 = black display and superslow modesets Linux cuda , ubuntu	21	24175	December 6, 2022
Omniverse Crash running USD demo: Cannot create cuda external memory for resource Synthetic Data Generation (SDG)	5	165	September 6, 2024

Kit App Crashes with Headless SSH Linux Config (GLFW Errors)

Related topics