Installing cuda toolkit 11.2 returns "Existing installation of CUDA Toolkit 11.2 found" despite supposably completely removed cuda

I am trying to re-install cuda toolkit 11.2 on my Ubuntu 18.04 server. To remove the previous installation, I tried the following commands:

sudo apt-get --purge remove nvidia-*
sudo apt-get --purge remove "*nvidia*"
sudo apt autoremove cuda
sudo apt-get --purge remove "*cublas*" "cuda*" "nsight*" 
sudo apt-get update 
sudo reboot

However, after rebooting, when I tried to install cuda toolkit 11.2 using

sudo sh cuda_11.2.0_460.27.04_linux.run

It returns “Existing installation of CUDA Toolkit 11.2 found”. Also, running “nvidia-smi” and “nvcc -V” seems as if they are functioning properly, and it seems all files in the installation directory of the previous cuda have not been removed. How should I resolve this problem? Thanks!

You were trying to remove the driver (despite incomplete steps).

You were trying to remove CUDA (despite incorrect order of command).

Recommended steps:

  1. Check installed driver version with dpkg -l | grep nvidia
  2. Check installed cuda version with dkpg -l | grep cuda
  3. Check if current driver version is still compatible with cuda version you want to install (compiled compatibility table as a quick reference)
  4. In case of compatible driver, remove old version of cuda and install the target cuda version
  5. In case of incompatible driver, upgrade driver, remove old version of cuda and install the target cuda version