Isaac Sim on OVX stuck when switching to path tracing

I am running Isaac Sim 2022.2.0 inside a VM on an OVX server, using Teradici as a remote desktop solution. Working with my scene works only when using the Real-Time renderer. FPS is around 20-60.

image

As soon as I switch to path traced rendering, the viewport and the whole application is almost stuck, with only 0.2 FPS.

image

This is the output of nvidia-smi (we have a grid driver installed, as described in the Teradici docs):

testuser@rendering-ovx0-lin-dev-00:~$ nvidia-smi
Wed Mar 22 15:49:38 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.85.05    Driver Version: 525.85.05    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A40-24Q      On   | 00000000:02:00.0 Off |                  Off |
| N/A   N/A    P8    N/A /  N/A |      0MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
testuser@rendering-ovx0-lin-dev-00:~$

Thanks for any help.
Bruno

Hi @bruno.vetter - Someone from our team will look at the issue and get back to you.

Can you try updating the drivers to 525.89.02 if possible?
Please try the latest Isaac Sim 2022.2.1 and also see if this happens on the latest Create or Code app.
There could be already be a fix in the latest Kit 104.2.

Edit: I might have the wrong driver downloaded, let me check…

I try to install the driver, but the installation will not finish:

ERROR: Unable to load the kernel module 'nvidia.ko'.  This happens most frequently when this kernel module was built against the wrong or improperly configured kernel sources, with a
         version of gcc that differs from the one used to build the target kernel, or if another driver, such as nouveau, is present and prevents the NVIDIA kernel module from obtaining
         ownership of the NVIDIA device(s), or no NVIDIA device installed in this system is supported by this NVIDIA Linux graphics driver release.                                          
       
         Please see the log entries 'Kernel module load error' and 'Kernel messages' at the end of the file '/var/log/nvidia-installer.log' for more information.

Is there actually a grid driver 525.89.02? I could not find any download source.

Try the 525.116.03 drivers
See Unix Drivers | NVIDIA
Remove the folders below before installing the drivers:

/etc/vulkan
/usr/share/vulkan