We installed AODT on our local server L40*8, with the driver version 550.107.02 on Ubuntu 22.04.5.
However, after running AODT, when trying multiple different USD format 3D images, they appear gray instead of in color.
Additionally, we discovered that OpenGL reported an error, even though we are using the recommended driver version 550.107.02 as specified in the documentation.
2025-01-07 01:03:37 [Error] [carb.glinterop.plugin] OpenGL Interop is not available. Upgrade your driver to latest for this feature.
2025-01-07 01:03:37 [Warning] [gpu.foundation.plugin] ECC is enabled on physical device 0
2025-01-07 01:03:37 [Warning] [gpu.foundation.plugin] IOMMU is enabled.
2025-01-07 01:03:37 [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 0 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 1 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 2 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 3 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 4 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 5 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 6 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:38 [Warning] [rtx.neuraylib.plugin] [IRAY:RENDER] 1.1 IRAY rend warn : CUDA device 7 (NVIDIA L40): ECC is enabled, this will reduce rendering performance (it is strongly recommended to disable ECC with iray)
2025-01-07 01:03:39 [Warning] [omni.kit.renderer.plugin] Reducing '/app/runLoops/present/rateLimitFrequency' from 120.000000 to primary monitor's refresh rate of 60.000000 ('VNC-0', '/output:59/crtc:58')
2025-01-07 01:09:46 [Warning] [omni.usd] Warning (secondary thread): in _ReportErrors at line 2890 of /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp -- In </Environment/sky>: Could not open asset @https://omniverse-content-production.s3.us-west-2.amazonaws.com/Assets/Skies/2022_1/Skies/Dynamic/ClearSky.usd@ for reference introduced by @omniverse://omniverse-server/Users/aerial/plateau/tokyo.usd@</Environment/sky>. (instantiating stage on stage @omniverse://omniverse-server/Users/aerial/plateau/tokyo.usd@ <0x7abadc008620>)
2025-01-07 01:09:46 [Warning] [omni.usd]
2025-01-07 01:18:22 [Warning] [omni.kit.window.property.templates.simple_property_widget] AttributeWidget.build_items took 0.20743298530578613 seconds
@user112062 Can you please attach:
- Screenshot of the map
- output of nvidia-smi
The map looks like this.
GPU is
nvidia-smi
Wed Jan 8 13:08:18 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02 Driver Version: 550.107.02 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L40 Off | 00000000:17:00.0 Off | 0 |
| N/A 45C P0 170W / 300W | 6393MiB / 46068MiB | 58% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA L40 Off | 00000000:2A:00.0 Off | 0 |
| N/A 35C P0 81W / 300W | 784MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA L40 Off | 00000000:3D:00.0 Off | 0 |
| N/A 34C P0 79W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA L40 Off | 00000000:63:00.0 Off | 0 |
| N/A 35C P0 78W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 4 NVIDIA L40 Off | 00000000:99:00.0 Off | 0 |
| N/A 34C P0 78W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 5 NVIDIA L40 Off | 00000000:AB:00.0 Off | 0 |
| N/A 35C P0 77W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 6 NVIDIA L40 Off | 00000000:BD:00.0 Off | 0 |
| N/A 34C P0 79W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 7 NVIDIA L40 Off | 00000000:E1:00.0 Off | 0 |
| N/A 34C P0 79W / 300W | 354MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 8091 C+G ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 6339MiB |
| 1 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 5078 C ./aodt_sim 424MiB |
| 1 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 2 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 2 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 3 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 3 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 4 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 4 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 5 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 5 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 6 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 6 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
| 7 N/A N/A 1770 G /usr/lib/xorg/Xorg 4MiB |
| 7 N/A N/A 8091 C ...cal/share/ov/pkg/aodt-1.1.0/kit/kit 332MiB |
+-----------------------------------------------------------------------------------------+
@user112062 Thanks. We have reproduced a similar issue and will update as soon as we have a solution.
1 Like
@user112062 Hi, since you are using L40 GPUs, Can you please check the current GPU mode on your setup using the “NVIDIA Display Mode Selector Tool”?
You can download the tool from here: NVIDIA Display Mode Selector Tool | NVIDIA Developer
Also, please make sure to use the RTX6000 driver, and not the L40 driver.
