Linux, CUDA, PyTorch: "Found no NVIDIA driver on your system"

We have a Linux System with the following setup:

  • NVIDIA driver version: 460.73.01
  • CUDA Version: 11.2
  • We have tried these images: nvidia/pytorch:20.11-py3, nvidia/pytorch:21.07-py3, see NVIDIA images.

As a test case, we run the following, which fails (trace is from nvidia/pytorch:20.11-py3):

Python 3.6.10 |Anaconda, Inc.|
[GCC 7.3.0] on linux
>>> import torch
>>> torch.cuda.is_available()
/opt/conda/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
False 

We have tried different images, for example gpu-jupyter, which do work. Why is it that the images directly supplied by NVIDIA fail on our system?