Record ram usage and gpu mem usage

[JeremyYuan]

A question is asked here.

Record ram usage and gpu mem usage

Hi, I’d like to record ram usage and gpu mem usage in linux system when inferring a plan file, Is there a solution to do so? I’ve used vmRSS in /proc/self/status and cudaGetMemInfo api to calculate this. I do this in the following process. enter main, record ram usage of current process and gpu mem currently used in the system. deserialize a planfile and run a inference. do the same thing with step 1. compute gpu mem usage diff of step 3 and 1, which includes ram usage. so the ram usage for…

Any advice is welcome. thank you.

[kayccc]

For Jetson platform, please refer to Tegrastats Utility doc: Tegrastats Utility — Jetson Linux
Developer Guide 34.1 documentation (nvidia.com)

[JeremyYuan]

Hi, kayccc, thank you for you info.
But what if I’d like to know specific process resource consumption(RAM and GPU mem), I mean I want to record specific model ram & gpu mem consumption when inferring, The more accurate the better.
So are there any APIs to be called for this purpose?

[AastaLLL]

Hi,

You can find an example below:

CUDA memory release

Hi, Please check this sample: #include <cuda_runtime.h> #include <iostream> #include <sstream> #include <fstream> #include <opencv2/core/core.hpp> #include <opencv2/highgui/highgui.hpp> #include <opencv2/imgproc/imgproc.hpp> #include "NvInfer.h" #include "NvCaffeParser.h" #include <cublas_v2.h> #include <cudnn.h> #define INPUT_BLOB "data" #define OUTPUT_BLOB "prob" #define ONE_MBYTE (1024*1024) class Logger : public nvinfer1::ILogger { void log(Severity severity, const char* msg) overr…

Thanks.

[JeremyYuan]

Thank you @AastaLLL for your quick response.
From the link you provide to me, I see we can get gpu mem free & all gpu mem in the system via cudaGetMemInfo API just as I said from the link I post.

@Record ram usage and gpu mem usage

Hi, I’d like to record ram usage and gpu mem usage in linux system when inferring a plan file, Is there a solution to do so? I’ve used vmRSS in /proc/self/status and cudaGetMemInfo api to calculate this. I do this in the following process. enter main, record ram usage of current process and gpu mem currently used in the system. deserialize a planfile and run a inference. do the same thing with step 1. compute gpu mem usage diff of step 3 and 1, which includes ram usage. so the ram usage for…

Now the question is how can I get mem usage in view of RAM. do I get the right way to query RAM usage.
Thanks.

[AastaLLL]

Hi,

Jetson is a shared memory system, which indicates that the physical memory is used by both CPU and GPU.
So the available memory for CUDA will be similar to the physical memory that remains.

Thanks.