How to get TensorRT model Energy consumptionn of AGX Orin?

[zijie-tian]

I need to obtain the GPU’s energy consumption during the inference of TensorRT models on AGX Orin. However,

1. There is no NVML library on AGX, so I cannot use methods like Triton to obtain power consumption.
2. I am worried that if I use trtexec to execute and utilize tegrastats to track energy consumption, it might include
   additional energy consumption overhead (such as model loading, etc.).

How can I get an accurate measure of energy consumption on AGX Orin?

[AastaLLL]

Hi,

We don’t have an application-level power monitor tool.
Please use tegrastats tool to get the info about the power.

$ sudo tegrastats

When testing with trtexec, you can set a large iteration to allow you to capture the power consumption of the inference step.

Thanks.