I would like to measure the maximum power consumption of the Thor DriveAGX platform by applying the highest possible load on the Thor chipset. My goal is to utilize all available resources within the chip, including CPU, GPU, and DDR bandwidth, while monitoring power consumption.
I have the following questions:
I am running a simple CUDA matrix operation, and tegrastats reports GR3D_FREQ at 99%. However, the GPU temperature hardly increases. How can I verify whether the GPU is actually under full load?
As I understand, Thor can deliver up to approximately 1,000 TOPS of INT8 performance. If I only run the CUDA matrix workload and observe a high GR3D_FREQ value in tegrastats, does that mean the GPU is being loaded close to its maximum INT8 processing capability?
Does NVIDIA provide any reference test applications or benchmark tools for conducting this type of heavy-load and maximum power consumption testing?
By the way, could you please let me know about Q2 which GR3D_FREQ and full TOPS support?
Just to clarify, if GR3D_FREQ is 70%, would it be reasonable to interpret that as roughly 70% utilization of the available INT8 performance (e.g., about 700 TOPS out of 1000 TOPS)?
It can be like if CPU/GPU utilisation is 90%, it indicates, out of 100 samples that are collected, 90 samples indicates CPU/GPU is used.
Could you also check User Guide — Nsight Systems helps?
Since my primary goal is not only performance benchmarking but also measuring power consumption under full load, I additionally ran FP8 GEMM tests using CUTLASS.
According to the Nsight results, both tests maintained a GPU clock of 1.54 GHz, while GR Active remained at 99.9% throughout execution.
Given these results, I would like to confirm whether I am effectively driving the Thor GPU to its maximum performance capability. If so, would it be reasonable to consider these benchmarks as achieving the full INT8 or FP8 1,000 TOPS performance that Thor is specified to deliver?
Below are the results from the FP8 GEMM test, showing how many GEMM iterations were completed per second.
The test was performed using the following CUTLASS example:
The purpose of this measurement was to monitor the sustained execution rate of the GEMM workload under continuous load conditions and to correlate it with power consumption and GPU utilization metrics.
I understand that there is a comment indicating that the examples under cutlass/examples are not suitable for benchmarking. Nevertheless, I would like to confirm whether it is expected that the performance converges to approximately 120 TFLOPS, as shown below.