I want to measure the power consumption of an application on ubuntu which finishes in just 250 ms on Quadro P4000.
Now for this I start nvidia-smi in the background and run the application on the forground.
Content of my power_measure.sh file:
nvidia-smi --query-gpu=power.draw --format=csv --loop-ms=10
Now I issue the command:
#./power_measure.sh & ./myapp
Now if I use high-resolution measurement i.e. --loop-ms is 5 to 50 ms then myapp does not gets chance to run.
If I use lower resolution say 100 or 200 ms then I do not get prints for power between myapp print logs.
How do I use nvidia-smi to continuously measure the power for an application. Thank you.
If I understand the description correctly, the problem may be not solvable with software-based tools. This is a common issue with software probes that when they are used at high frequency they start changing the system behavior and disturb the very same metric(s) they are trying to measure.
The first thing I would suggest is to lengthen the run time of your app. That may be trivial or near impossible, depending on how the app works. The trivial case would be to just repeat the core portion of the app, say 100 times. That gives you the steady-state power draw, that you can sample at, say, one-second intervals to keep intrusion effects to a minimum.
If you need the instantaneous power draw of the unmodified app, I would suggest a hardware-based solution, where you intercept relevant power supply pins of the PCIe slot as well as the PCIe power cables (if any). I am not qualified to go into specific details, but I have seen such setups work in practice (in HW lab settings).