if several arrays are too large to be loaded on GPU, will cuda swap some memory not in use? Thanks!
No. You will receive an out of memory error if you cudaMalloc more than the available free memory.
This is sort-of inconvenient, as back in the old GL days, you could easily malloc and populate much more textures than onboard vidmen, and the driver would page things in and out for you. This was (and still is on old HW) very convenient. We even had a technique of texturing a “dummy quad” which forced the driver to page exactly the amount of data into vidmem that we needed for the next expensive computation :)
All hacky, all very obscured due to mapping algorithms to graphics primitives, but…
It’s a trade-off between convenience and performance. Though I’m sure you’d consider it very inconvenient if your kernel all of a sudden got only 3 GB/s memory throughput as opposed to 70 GB/s, simply because data was moved to host RAM, and you had no control over it whatsoever :)
perfectly right. We just have to implement our own paging policy – tailored to the application – these days to get good performance ;)