How can I prevent Xorg from taking control of a graphic card?

I have two cards, a primary one, used for the desktop, and a secondary one, used for other jobs; both use the Nvidia proprietary driver (“nvidia” module), from the Nvidia PPA.

I need the second card not to be controlled by Xorg, however, I can’t find a way to accomplish this.

This is my Xorg configuration:

Section "ServerFlags"
	Option "AutoAddGPU" "false"

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusId          [...] # bus id of primary GPU

Section "Screen"
    Identifier     "Screen1"
    Device         "Device0"
    Monitor        "Monitor1"
    DefaultDepth    24
    Option         "Stereo" "0"
    SubSection     "Display"
        Depth       24

In theory, the AutoAddGPU option should accomplish this; in practice though, Xorg still controls the card: I can’t unbind the nvidia module until X is shutdown, but also, I can see from the Xorg log:

# X boot (extract):

[    15.516] (II) NVIDIA(GPU-1): NVIDIA GPU NVIDIA GeForce RTX [...]
[    15.516] (II) NVIDIA(GPU-1):     PCI:[...] (GPU-1)
[    15.516] (--) NVIDIA(GPU-1): Memory: [...]
[    15.516] (--) NVIDIA(GPU-1): VideoBIOS: [...]
[    15.516] (II) NVIDIA(GPU-1): Detected PCI Express Link width: 16X

# X shutdown (extract):
[    29.088] (II) NVIDIA(GPU-0): Deleting GPU-0
[    29.088] (II) NVIDIA(GPU-1): Renaming GPU-1 to GPU-0
[    29.088] (II) NVIDIA(GPU-0): Deleting GPU-0

If I use a stub driver for the card, Xorg does not take control (there is no GPU-1 mention in the log), however, I wanted to know if it’s possible to still retain the nvidia driver, but have Xorg not interact with it at all.