Hi
I would like to know if it is possible to get the only GPU memory usage information? as I know GPU and CPU on Orin share the same memory over memory controller, I used torch.cuda.mem_get_info to get the memory consumption, but the information I got from this function is the same as one I got via htop/free -h.
I am wondering torch.cuda.mem_get_info will return GPU+CPU memory usage information, there is no possibility to get only GPU memory consumption. is this correct?
Correct.
FYI,
cudaMemGetInfo() returns the available memory amount for GPU.
Also, tegrastats RAM parameter shows the overall used memory.
Since Tegra(DRIVE Orin) is a shared memory system, we don’t separate the memory usage between CPU and GPU.
Hi @SivaRamaKrishnaNV
thanks for your prompt answer, I have the second question :
If I use this :
// Allocate device memory
cudaMalloc((void**)&d_a, sizeof(float) * N);
how does this get the memory? does this like this flow:
cudaMalloc((void**)&d_a, sizeof(float) * N); -->CUDA LIB–>Linux kernel->MMU–>GPU
or
cudaMalloc((void**)&d_a, sizeof(float) * N); -->CUDA LIB–>Linux kernel->GPU–> system memory
Dear @MicroSW,
May I know why you are looking for implementation workflow details of cudaMalloc?
The implementation details of APIs are not shared publicly.
I don’t need to understand in detail how this api implemented, I only want to know how the memory is managed in Orin when I run a CUDA application, such as SLM. In my opinion, GPU has its own memory space, but in this Orin, seems not.
the purpose is that I am working on memory trace, and want to develop a simple tool to trace GPU workload on memory including Read/Write.
The call path is like cuda API-> nvrm lib-> linux kernel → MMU-> GPU
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.