9800GX2 as 2 GPUs under Linux

I’m trying to run a GX2 so that I have 2 separate X servers, each running one screen, and each using one GPU. How can I achieve this? What do I need to put in my xorg.conf(s) to specify a different GPU from the GX2 for each X server instance?

lspci reports this:
03:00.0 3D controller: nVidia Corporation Unknown device 0604 (rev a2)
04:00.0 VGA compatible controller: nVidia Corporation Unknown device 0604 (rev a2)

I cannot seem to set device 3 (via BusID option in xorg.conf) - nothing comes up on the screen on either of the 2 ports. BusID PCI:4:0:0 works fine as before, though. How do I get an X instance to run exclusively on each GPU?

I’m trying to do this in order to run 2 instances of the folding client under WINE, so would need Cuda support on each instance of the X server.

TIA.

[edit: lspci actually reports more devices that might be relevant:
01:00.0 PCI bridge: nVidia Corporation Unknown device 05be (rev a2)
02:00.0 PCI bridge: nVidia Corporation Unknown device 05be (rev a2)
02:02.0 PCI bridge: nVidia Corporation Unknown device 05be (rev a2)
03:00.0 3D controller: nVidia Corporation Unknown device 0604 (rev a2)
04:00.0 VGA compatible controller: nVidia Corporation Unknown device 0604 (rev a2)
]

OK, I managed to get further, but I am seeing massive performance degradation when running two instances of a cuda application (in this case folding client under WINE).

Running 1 F@H client and 1 X server (same user, obviously), performance is about 90s/%

Running 1 F@H client and 2 X servers (different users), performance drops to about 130s/%

Running 2 F@H clients and 2 X servers (same user for both), performance drops to 287s/% (both seem to run on the same GPU)

Running 2F@H clients and 2 X servers (differentusers), isn’t quite as bad, but the two clients seem to get unbalanced in terms of performance, and they still between them don’t reach the throughput of a single client (~100s/% and ~150s/%)

It almost feels like there is either a shared memory stomp somewhere that makes both cuda instances bind to the same GPU or a very expensive context switch happening somewhere when running as different users, even though each instance is bound to a separate CPU, and a separate X server running in a separate GPU.

CUDA doesn’t provide any way for applications to tell which GPUs are used and which aren’t. Thus: unless there is some inter-process communication between the two applications you are running, it is very possible that they are running on the same GPU. You can check by running /usr/sbin/lsof /dev/nvidia* . nvidia1 is the first GPU and nvidia2 is the 2nd GPU. Processes that are using the GPU for CUDA open the file with the “mem” descriptor (I’m working on writing a gputop program using this, but it isn’t ready for the public yet…)

The other possibility is that the application is bound by the PCI-e bandwidth to/from the card. Since the 2 9800 GX2 cards shared the same PCIe slot, performance will be reduced with two processes competing for bandwidth.

Thanks for your answer. In the end it turned out that both instances were binding to the same Cuda device ID. The whole exercise of running two separate X servers (which I managed to get working eventually anyway) was unnecessary. In the end, it was a simple case of changine the device ID in the wrapper library.

Thanks for responding, though. :-)

Here’s the thread with more details in case anyone in the future bumps into a similar issue:
http://foldingforum.org/viewtopic.php?f=52&t=3744&start=75