System Memory Fallback for Linux

Graphics / Linux Linux
1

Have a model that is throwing CUDA out of memory errors on my development box with a 16GB 4080. I can find information on System Memory Fallback on Windows to fallback to system memory, however I am unable to find any information on how to do this in Linux.

Any help would be appreciative.

This error is being thrown when running a python script.

2

Sorry, not supported by the nvidia linux driver.

3

Would you happen to know if this support is roadmapped for Linux? It’s pretty useful for running inference closer to device maximums without worrying so much about killing the job.