I hear my GPU can use system RAM...

Accelerated Computing CUDA CUDA Programming and Performance
1

I’ve heard that GPUs can run threads off of system RAM assuming the dedicated memory is already being occupied. Can I do this in CUDA?

2

Look into the UVA, but it slow, since the memory is access over the pci

3

See pages 27-29 of the CUDA C Programming Guide.

It works really well. With careful coding, a kernel can saturate the PCIe bus when reading from write-combined memory.