You cannot use any of those CUDA releases. They are designed for PCI GPUs, and the Jetson GPU is integrated directly to the memory controller. You will need to use the one which comes from the same JetPack/SDKM release as is on the Jetson. To see that release you can use one of these:
head -n 1 /etc/nv_tegra_release
dpkg -l | grep 'nvidia-l4t-core'
If you can’t remove the other driver you may need to flash again.
No, the SD card image will not have this. However, it will have the repository setup completed such that you can install via the “sudo apt-get install ...” mechanism over the internet. What do you see from: apt search cuda
I was not aware the prebuilt image has CUDA. The “apt” mechanism would still work, but sounds like it is unnecessary.
Keep in mind that the GPU driver is not the CUDA support library. The install of the incorrect PCI based GPU driver may interfere (I don’t know what the consequences of installing this will be). There was never any need to install the GPU driver, this was itself always there. The CUDA software does depend on the GPU driver, but you never had the CUDA software and so there was no chance the GPU driver could succeed with CUDA. You might consider starting with a fresh SD card install, and then going to the “apt search cuda” command to see what is available. Do experiment with finding CUDA via apt before you start over.
To clarify, yes if you flashed the SD card image, then the drivers and CUDA Toolkit are already installed (under /usr/local/cuda). These are for using Jetson’s integrated GPU.
nvidia-smi isn’t supported on Jetson, so even with the CUDA drivers installed, there won’t be nvidia-smi.