NVIDIA A16 performance drops under multiple concurrent workloads

Hi everyone,

I’m testing an NVIDIA A16 in a virtualized environment with multiple concurrent workloads, and I’m seeing occasional performance drops even though overall GPU utilization is not close to 100%.

Has anyone experienced similar behavior with the A16? I’d like to know which metrics I should check first, such as PCIe bandwidth, GPU memory, encoding load, or vGPU configuration.

Are there any recommended settings or monitoring steps for troubleshooting performance drops under concurrent workloads?

Thanks!