Our lab has got a new DGX with A100-SXM4-40G cards. It is running nvidia driver ver. 450.80.02 on UBUNTU LTS 20.04. There is no cuda toolkit preinstalled. I require it to work with Pytorch.
So I am trying to install the latest compatible cudatoolkit(11.03 as per Nvidia) by adding the official repository in APT. I am following the official installation procedure but my installation is always getting stuck at following:
.
.
.
.
Preparing to unpack …/630-xserver-xorg-legacy_2%3a1.20.13-1ubuntu1~20.04.3_amd64.deb …
Unpacking xserver-xorg-legacy (2:1.20.13-1ubuntu1~20.04.3) …
Errors were encountered while processing:
/tmp/apt-dpkg-install-pnkesj/226-nvidia-compute-utils-515_515.65.01-0ubuntu1_amd64.deb
/tmp/apt-dpkg-install-pnkesj/231-nvidia-utils-515_515.65.01-0ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
.
.
…
After this the package is broken and nvidia-smi, nvcc, etc. dont work.
The cudatoolkit in apt repository is very old v10.1 and somehow the latest pytorch is unable to access the GPU with that ver.
Please provide any solution!!!
DGX A100 has dedicated repos and Ubuntu OS for managing its drivers and various software components such as the CUDA toolkit.
Please refer to the DGX system user guide chapter 9 and the DGX OS User guide.
System user guide: https://docs.nvidia.com/dgx/pdf/dgxa100-user-guide.pdf
OS user guide: https://docs.nvidia.com/dgx/dgx-os-5-user-guide/index.htm
You might have to clean up the system by removing the non-dgx repos and uninstalling incompatible drivers and software components, and then re-installing the correct drivers (remember to install fabric manager on DGX A100). Again, please refer to the docs above.
For CUDA toolkit refer to this section:
https://docs.nvidia.com/dgx/dgx-os-5-user-guide/index.html#det-new-avail-tk-rel
You should have the following apt sources files with content:
/etc/apt/sources.list
deb http://security.ubuntu.com/ubuntu focal-security main multiverse universe restricted
deb http://archive.ubuntu.com/ubuntu/ focal main multiverse universe restricted
deb http://archive.ubuntu.com/ubuntu/ focal-updates main multiverse universe restricted
/etc/apt/sources.list.d/dgx.list
deb http://repo.download.nvidia.com/baseos/ubuntu/focal/x86_64/ focal common dgx
deb http://repo.download.nvidia.com/baseos/ubuntu/focal/x86_64/ focal-updates common dgx
/etc/apt/sources.list.d/cuda-compute-repo.list
deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /
To run PyTorch, consider using docker containers. PyTorch in particular is available here: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch