As previous threads has mentioned that nvidia-smi is available but it doesn’t log anything, making it hard to monitor the gpu usage for my application. Is there any news on if support is coming? And also if there is any alternatives to use instead.
I run the docker container:
nvcr.io/nvidia/deepstream:8.0-triton-multiarch
Kind Regards
You should be able to see the GPU utilization, but you can’t see a break down of the RAM usage.
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00 Driver Version: 580.00 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA Thor Off | 00000000:01:00.0 Off | N/A |
| N/A N/A N/A N/A / N/A | Not Supported | 94% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 5243 C VLLM::EngineCore 0MiB |
+-----------------------------------------------------------------------------------------+
You can also run nvidia-smi dmon if you want a break out of the processing across the GPU (sm), video encoder (enc), video decoder (dec), jpg, and the optical flow accelerator (ofa)
# gpu pwr gtemp mtemp sm mem enc dec jpg ofa mclk pclk
# Idx W C C % % % % % % MHz MHz
0 - - - 0 0 0 0 0 0 - -
0 - - - 0 0 0 0 0 0 - -
0 - - - 0 0 0 0 0 0 - -
0 - - - 0 0 0 0 0 0 - -
0 - - - 0 0 0 0 0 0 - -
0 - - - 0 0 0 0 0 0 - -
0 - - - 92 0 12 1 0 0 - -
0 - - - 92 0 23 1 0 0 - -
0 - - - 91 0 20 1 0 0 - -
0 - - - 92 0 7 2 0 0 - -
0 - - - 92 0 19 0 0 0 - -
0 - - - 91 0 22 0 0 0 - -
0 - - - 93 0 10 1 0 0 - -
0 - - - 92 0 14 2 0 0 - -
0 - - - 89 0 22 3 0 0 - -
0 - - - 91 0 23 2 0 0 - -
0 - - - 92 0 21 1 0 0 - -
0 - - - 91 0 23 0 0 0 - -
0 - - - 91 0 10 1 0 0 - -
0 - - - 93 0 16 0 0 0 - -
0 - - - 90 0 18 1 0 0 - -
I believe the jtop utility needs to be compiled for Thor? I don’t know if it provides any additional insights either. Another command: tegrastats will give you usage, power, & temperature readings.
sudo apt install nvtop
Does not show “gpu” memory. Does show gpu related process memory use.
Nvtop is a ncurses-based GPU monitoring interface which provides information
on the GPU states (GPU and memory utilization, temperature, etc) and well as
information about the processes executing on the GPUs.
Here’s instructions to install jtop for Thor which shows some gpu memory information.
Hi,
GPU usage is supported in nvidia-smi.
Could you share more information about the feature you need?
Thanks.