I have completely removed all previous NVIDIA drivers and cuda installations, and then installed cuda.12.2 together with its GPU driver 535.104.05 (for RTX 4090) on Ubuntu 22.04, and PyTorch has no problems finding and using the GPU.
However, when installing cuQuantum (cuquantum-local-repo-ubuntu2204-23.06.1), I get the following error:
The following packages have unmet dependencies:
libcuquantum0-cuda-11 : Depends: libcublas.so.11 but it is not installable
libcuquantum0-cuda-12 : Depends: libcublas.so.12 but it is not installable
E: Unable to correct problems, you have held broken packages.
I can see that libcublas.so.12 (linked with libcublas.so.12.2.1.16) is in /usr/local/cuda/lib64, the path of which has been added to LD_LIBRARY_PATH (and to make sure it is available to get-apt, I even used -E to pass the environment in to sudo). I have also tried to install to the entire HPC package (in /opt, which has the same version of libcublas.so.12), and added its path to LD_LIBRARY_PATH, but it make no difference.
Note that there was a similar problem recently with libcublas.so.11, which was resolved by fixing the NGC service on NVIDIA side. Not sure if this may be a similar issue?