I am a rookie to CUDA. Here are questions I have in the project. cudaMalloc(&d_x, N*sizeof(float)) By cudaMalloc,it should be on the device? printf("Address: %p\n", d_x) output is: 0x7fad1b200000.
My question is:
Is this address virtual or physical? If virtual, how could I based it get the physical address?
If I don’t call cudaFree, will this data keep residing here, or cudaFree will be called automatically?
Is it possible to store parameters of the model to somewhere of the global memory after the end of training/inference?
The data will keep residing there until the owning process terminates (at that point “cudaFree is called automatically”).
Sure, typical frameworks like TF and pytorch give the user the ability to locate a model on host or device, and move the model from host to device or vice versa. I suggest asking questions about how to do things in frameworks in other forums, like this one for pytorch.
Hi Robert,
I comes with two extra questions, hope it won’t take you too long.
If I enable the MIG for A100 and create two 2-gpcs instances, g1 and g2, do they (g1,g2 and the rest part of A100) also share the virtual memory space?
In MIG documentation, P2P is disabled under MIG mode, any communication between instances, like g1 and g2, has to go through the host memory? What about the communication for g1 and the rest of A100?
Every processor in the system shares the same unified virtual address space, when UVA is in effect. UVA is in effect any time you are using an A100 GPU, regardless of usage of MIG or not, or any other possible variation in how you may be using the A100.
Yes. (It is possible to do CUDA IPC between compute instances, but ordinary cudaMemcpyXXX operations between MIG instances would not follow a P2P path.)
When you enable MIG mode, you must define MIG instances for any usability of that A100. It is not possible to use MIG instances but somehow leave the “rest of A100” for “ordinary use”. The only thing that gives you any functionality at that point are defined MIG instances. Undefined space on the A100 is not usable, when MIG mode is enabled. Once you enable MIG mode, P2P is not supported for any sort of usage of any part of the A100.