Got out of memory from cudaMemcpy

Accelerated Computing CUDA CUDA Programming and Performance
9

I see the same issue on my CUDA 11.3 machine. I suggest filing a bug.

You already have a possible workaround; do cudaSetDevice() for the relevant pointer before the cudaMemcpy() operation.