How to get CUDA container to utilize graphics card drivers

I am using: nvcr.io/nvidia/cuda:latest. After installing the cuda toolkit it says it cant find a driver and nvidia-sim doesn’t work

You need to follow the directions here:

https://docs.nvidia.com/ngc/ngc-titan-setup-guide/index.html

to set up your own local machine to use docker containers with the nvidia-docker runtime plugin.

You should then launch the container specifying the nvidia-plugin for docker.

That is the only thing you need to do. If you intend to install a new or different CUDA toolkit in the container (shouldn’t be necessary), you need to make sure that the CUDA toolkit you install is compatible with the driver in your base machine.

Why are you installing a cuda toolkit in the cuda container? cuda:latest already has the latest (10.1) cuda toolkit in it:

https://ngc.nvidia.com/catalog/containers/nvidia:cuda/tags

Also, note that there are forums specifically for NGC questions.
After discovering that you posted there as well:

https://devtalk.nvidia.com/default/topic/1051603/container-cuda/cuda-amp-nvidia-version-for-googles-container-optimized-os/

I see that you seem to be using GCP and you don’t have a GPU driver installed at all. In that case you should follow the GCP setup instructions:

https://docs.nvidia.com/ngc/index.html#ngc-with-google-cloud-platform-(gcp)

https://docs.nvidia.com/ngc/ngc-gcp-setup-guide/index.html

Im installing the toolkit to also install the driver. Also the gpu that I am using is an Nvidia Tesla V100. After following GCP setup instructions, Once with Container-optimized OS and the other with Ubuntu 18.01.0, I still could not get the nvidia cudas container to pick up the driver(the driver for COS wasn’t installed, and the driver for Ubuntu was).

The image you should be using is this one:

nvidia-gpu-cloud-image-20190219

https://docs.nvidia.com/ngc/ngc-gcp-vmi-release-notes/index.html

That has the 410.104 driver in it already.