Strange device memory behaviour

I have an algorithm using pthreads I am trying to distribute across the four devices in our S1070, but one of the devices, device 3, is behaving strangely.

When using device 3 on its own when I use cuMemGetInfo before allocating any memory it tells me it has no memory at all(!), but after memory allocation it has about 50% free and the algorithm runs to completion OK.

When I use device 3 as one of a pair the same thing happens.

But when I run all four devices device 3 tells me it has no memory before memory allocation, but after memory allocation it has 0.09% free (it should be 50% free like it is for the other three devices and like it should when device 3 is used on its own or part of a pair) and the algorithm crashes with a “out of memory” error message.

Does anyone have any explanations and/or remedies?

Thanks in advance.