Pinned vs. Unpinned memory

Is there a reason to use unpinned memory instead of pinned memory? I read somewhere about a problem with pinned memory and multiple GPUs. Is that the only issue or are there others? Is that still an issue?

Thanks

The main one is that your OS cannot page pinned memory; once you allocate pinned memory, that’s it, your program owns it until it frees it (or it terminates). As a result, if you try to allocate past 3.8GB or so of pinned memory on a 4GB machine, blammo, the app will be killed (gracefully on Windows, less gracefully on Linux because the kernel will just kill whatever asks for memory next, which could be anything).

I forget what the other caveats are right now. I want to say that only the creating thread can access the memory allocated by cudaMallocHost. I imagine this is the cause of the multi-GPU problems as well.

Other people have mentioned that even before you hit the limit, you’ll start to observe poor system performance when you allocate too much pinned memory.