Multi-GPU debugging

I’ve got a machine with a pair of C2050s in it along with a seperate capability 1.3 card driving the display.

According to the CUDA-gdb manual I should be able to do separate debugging on the cards using CUDA_VISIBLE_DEVICES.

Indeed I can so long as both applications start before either hits a breakpoint.

However as soon as one card has paused, no other CUDA jobs will start until it is resumed.

Is there a way around this?

Are you running an accelerated desktop off the 1.3 device? As an experiment, can you try switching down to runlevel 3 (no X) and retry your experiment?
Can you also post the actual arguments to CUDA_VISIBLE_DEVICES you’re using?