#2 Depends on your motherboard, usually there is a primary slot that gets initialized first with video, or there might be an option in the BIOS to select a slot that gets initialized first (more unlikely now with newer boards) You probably won’t be able to dynamically switch cards if that’s what you’re asking. Sometimes depending on the board, if you connect a video input it might switch as active, but it’s not common, and usually determined by slot position in later motherboards.
(1) It depends on what you mean by ‘use’. The primary card as determined by your motherboard will show video, and will be the card you can plug in display devices to. I’m not familiar if one would be able to use two cards to drive 2 or more different screens if that’s what you’re asking. If you mean CUDA-wise, both cards should be ‘available’ by default for use by CUDA programs… if deviceQuery sees it, it’s available for use. To use both at the same time requires corresponding logic in your CUDA code.
(2) If you’re able to disable a physical slot from your BIOS, then yes, the card in that slot will show up to the O/S, thus meaning it will be like it is ‘invisible’, I believe disabling a slot actually does just that, sets an option that physically does not send power to the slot. Even if you can’t disable the slot, the CUDA_VISIBLE_DEVICES environment variable lets you make a card ‘invisible’ as far as CUDA programs are concerned. If you run deviceQuery after setting that variable to hide a GPU, you’ll see it will not show up.
(3) Not sure what ‘CUDA mode’ implies here. If a CUDA compatible card is installed, and you’ve already configured the toolkit, CUDA will be available always. Again, if you want to selectively ‘disable’ cards from CUDA being used, just do it with the CUDA_VISIBLE_DEVICES environment variable.
As vacaloca is saying, there is very little difference between a CUDA-capable device running your display and a CUDA-capable device running CUDA programs. You can do both on the same card at the same time, with only a few drawbacks:
The GUI will use some of the device memory, so CUDA programs can’t use as much.
The device can’t update the GUI display while a CUDA kernel is running, so if you have long kernels, the display will be very jumpy. If the kernel goes past a few seconds, the driver watchdog will abort it to keep the display from freezing for too long.
There is some context switching overhead every time the card has to switch between running a CUDA kernel and updating the display, so if you are doing 3D rendering and running a CUDA program, you’ll see some slowdown.
To avoid these drawbacks, all you have to do is stop the X.org server or configure it (as eugeneo showed above) to use a different device for the display. No BIOS or kernel fiddling required.
(Note that if you boot Linux with no X.org server, the /dev/nvidia* entries used by CUDA programs might not be automatically created. The Linux release notes show what you have to add to your startup scripts to ensure that those device files are created at boot.)
“To avoid these drawbacks, all you have to do is stop the X.org server or configure it (as eugeneo showed above) to use a different device for the display. No BIOS or kernel fiddling required.”
That’s what I mean with “CUDA mode”.
It doesn’t feel good to each time log in from alternate terminal and shut down X server.
It could as well boot to runlevel 3 (was it?).
But then - good bye all X-based tools, like geany or ddd (Do I need them?).
If I had both X and CUDA running in use, then I need two cards?
You don’t need two cards, but there are advantages to having two cards. I have worked with systems where CUDA runs directly on the same GPU as X. It was fine, since I didn’t mind the restrictions I listed above.
Then what happens to the X if you are using one card and debugging CUDA program with a breakpoint set?
Anyway, the booting to “CUDA mode” is solved: it’s easy to do with grub-customizer
Maybe someone else finds this interesting too. It can also be used if there is something wrong with X.
Just add an entry, copy the stuff from the default entry and change the boot parameters
“quiet splash” to “text”, and finally save. That’s all.
I have follow the instructions to start X on one of my Graphics cards. I have four card (two Titan Black, one Titan X and one GeForce 970).
My motherboard actually have two PCI Express 3.0 X16 so I want to put one of the Titan Black in one of these PCI Express and the Titan X on the other one.
I have change the configuration succesfully in the /etc/X11/xorg.conf but after doing so, I can log as normal into the system, the nvidia-smi command shows the X server running on the correct graphics card but when I tried to switch to a tty console, I havent any, it seems that none of them start during the boot.