I am working with the following:
Ubuntu 22.04.4 LTS
Nvidia driver: 555.42.06
Nvidia RTX 3080 Ti
I have multiple cuda installations in the /usr/local folder but I have added the cuda-12.1 bin to PATH and lib64 to LD_LIBRARY_PATH
I am simply trying to run this repository: GitHub - nerfstudio-project/gsplat: CUDA accelerated rasterization of gaussian splatting
I have installed the CUDA12.1 pytorch binaries using pip, which based on my online research shows that it ships with the CUDA binaries to run it without external setup.
However, if I don’t add the /usr/local/cuda-12.1/bin to the PATH and try to run the project it throws a “gsplat: No CUDA toolkit found. gsplat will be disabled”
After adding to PATH, I dont get this error anymore, it starts initializing but fails after a while saying “RuntimeError: CUDA error: no kernel image is available for execution on the device”
Looking at the driver compatibility, it seems that the nvidia driver version 555.42.02+ is not compatible with any other CUDA except 12.5. Is this the reason the code is not running? Do I need a different nvidia driver? The github codebase I have linked is not just restricted to Pytorch but also has custom CUDA kernels that is linked to the python code.
Perhaps this is why I can do basic CUDA device hosted operations in pure pytorch but it fails when trying to run this repository. Any tips would be heavily appreciated as this is all new to me.