Performance/metrics of distributed learning

Dear ladies and gentlemen

For my bachelor thesis at the FFHS I intend to measure the performance gain of distributed training. My goal is to find a metric which allows to test if a certain NN problem would be faster in a multi-gpu environment. I intend to use two RTX 3060 GPU and tensorflow (mirrored strategy).

My instructor from the school suggested that I ask you directly as you would mostlikely already have papers on this topic. Since Nvidia is the leading producer of GPUs and is also having a research center for artificial intelligent, I was hoping that the company would have published any scientific papers regarding distributed learning. Those papers dont have to be specific for tensorflow or RTX 3060.

Best regards, Matthieu Riolo