We are investigating Tensor Core FP32-accumulation performance on an NVIDIA GeForce RTX 5060 Ti 16GB (GB206, SM120), running Linux driver 595.112.07.
Using matched native HMMA instructions, we observe approximately 1.95× longer execution times for HMMA.16816.F32 than for HMMA.16816.F16. A historical legacy RM readout reports FMLA32 = 3.
The installed NVIDIA driver also registers the read-only NV2080_CTRL_CMD_GR_GET_SM_ISSUE_RATE_MODIFIER_V2 (0x2080123C), but runtime support on GB206 has not been established.
Could NVIDIA confirm whether FMLA32 controls the issue throughput of HMMA.16816.F32 on GB206, and identify any officially supported read-only diagnostic interface, required permissions and authenticated configuration mechanism for that setting?
We are seeking documented, supported functionality only. We are not requesting a firmware exploit or a protected-register bypass.
Any clarification or relevant technical documentation would be appreciated.
Follow-up: RTX PRO 2000 Blackwell (GB206) comparison
A useful independent control would be a native Tensor Core throughput measurement from an NVIDIA RTX PRO 2000 Blackwell desktop GPU (GB206).
On our GeForce RTX 5060 Ti GB206, matched native HMMA testing shows FP32-accumulating instructions achieving approximately half the FP16-accumulating throughput.
Would an RTX PRO 2000 owner be willing to run a voluntary, stock-settings FP16-accumulation versus FP32-accumulation benchmark?
An existing public implementation is available here:
The useful results would be the GPU identity, driver version, per-SM-clock throughput ratio and raw benchmark output.
No firmware modifications, register access, clock overrides or hardware unlock attempts are requested. Please run a benchmark only if you have permission to use the GPU and are comfortable reviewing the code.
This comparison would help determine whether GB206 professional and GeForce products have different native Tensor Core accumulation throughput.
On Turing 7.5, Ampere 8.6 and Ada 8.9 (and I believe I also saw a forum post about Blackwell 12.0) the consumer and workstation GPUs (not talking about datacenter 8.0, 10.x) differ in their accumulation speed. For FP16 input: FP32 accumulate is half as fast as FP16 accumulate for the consumer GPUs. TF32 input with FP32 accumulate is further half as fast as FP16 with FP32 accumulate (so it also differs).
The performance numbers are (typically/often) published in the Nvidia datasheets of the ‘GPUs’ (the whole GPU may be from a 3rd-party supplier; so perhaps I should better call it GPU classes or GPU chips?) and in the architecture whitepapers.
Thank you, Curefab, for the explanation and for pointing me toward NVIDIA’s published specifications and architecture whitepapers.
This helps clarify our findings. On my RTX 5060 Ti (GB206, SM120), we measured approximately 1.95× longer execution times for native HMMA FP32 accumulation compared with FP16 accumulation. Your explanation confirms that this is consistent with the documented GeForce performance characteristics rather than necessarily an issue with my GPU.
For me, the practical result is that trying to unlock the legacy FP32-accumulation throughput may not be the most productive direction. We have already identified alternative optimization paths, particularly native Blackwell block-scaled FP4/FP8 Tensor Core instructions and improving GPU memory residency for local LLM inference.
One question remains interesting: whether the FMLA32 speed-select setting is directly responsible for this difference on GB206, and whether NVIDIA provides a supported read-only interface to inspect it.
Thanks again for helping clarify the hardware behavior. It gives us a much better direction for our optimization work.
The RTX PRO whitepaper shows the GB202 PRO 6000 achieving the same FP16/32 accumulation throughput, whereas the same chipped 5090 has only half performance.
It would seem reasonable the same relationship between PRO 2000 and 5060 may apply. One hopes the generous price premium gets you something… :)
I am glad I could help.
For the requested read-only interface a Nvidia official could answer.
As alternative as it is as far as I understood quite broadly across all recent desktop vs. workstation GPUs, any other parameter which differs between them could stand in. It could be ECC memory support, RDMA support, etc.
Another option could be to write a short performance benchmark, which is run at the start of your program. To compensate for frequency and other factors, you could compare the performance ratio between FP16 and FP32 accumulation.