A CUDA out-of-memory error can be returned in situations where the call would require establishment of a device context on a device that you are “not using” or that you have not yet established a device context on, and some other user/process is using that device, and probably has used up quite a bit of memory on that device.
This particular issue can only occur in a multi-GPU scenario, and can only occur when there is more than one user or process, using GPUs.
An example/write-up is here.