Dedicate one Nvidia card for CUDA and another Nvidia card for Xorg?

As the topic says, I have got a cheaper NVIDIA card that I would like to use for Xorg and an expensive NVIDIA card that I would like to use exclusively for CUDA computation. Thus far, whatever I tried, I was not able to prevent the expensive card to be used by Xorg as well, for example:
fuser -v /dev/nvidia[0-1]
shows Xorg, x11 and akonadi on both cards which is confirmed by nvidia-smi showing a small amount of memory (2MB) without doing anything on the expensive card.

Is there a way to achieve this?

EDIT: just noting that I do not care much about the 2MB but from some reason, the expensive card gets “stuck” at Xorg from time to time (showing near 100% utilization and fan running at 100% endlessly)

Please try
Option “AutoAddGPU” “false”
in a Serverflags Section and
Option “ProbeAllGpus” “false”
in the nvidia device section.

Many thanks for your reply, unfortunately, that did not help. I have been actually using “AutoAddGPU” “false” already and the second option did not change anything - there are still 2MB used by x11 and related processes on the CUDA card, fuser still reports kwin_x11, plasmashell etc. These are the relevant sections of my config:

Section “ServerFlags”
Option “AutoAddGPU” “off”

Section “Device”
Identifier “Device0”
Driver “nvidia”
VendorName “NVIDIA Corporation”
BoardName “GeForce GTX 1650”
BusID “PCI:66:0:0”
Screen 0
Option “RegistryDwords” “RMUseSwI2c=0x01; RMI2cSpeed=100”
Option “PrimaryGPU” “on”
Option “ProbeAllGpus” “false”

Please run as root and attach the resulting nvidia-bug-report.log.gz file to your post.

Attached. EDIT: attachment removed

Xorg is only running on the 1650, so the config options are fine. You should configure and start nvidia-persistenced though, to keep the 1080 alive.

Oh, fine then, many thanks. I thought that fuser showing kwin_x11 etc on both nvidia0 and nvidia1 means that both cards are used by it… but I can see a difference of mmaped mode and I am not sure whether this was the case the last time my X was frozen by the 1080 misbehaving (which I want to prevent). So for the moment I’ll just see whether it’s indeed stable now and look at your suggestion of nvidia-persistenced.

Of course, also set CUDA_VISIBLE_DEVICE to not interfere with xorg.

If anyone with the same problem reads this, it turns out that none of the suggestions thus far solved the problem. It’s still not completely dedicated - Xorg is still somehow connected to both cards and if an issue happens, it’s still impossible to reset the CUDA card without Xorg restart.

I have opened a new thread about it (thus far without a solution but if anything is discovered, I’ll write it there):