K80 with Deep Learning

I recently purchased a K80, I was planning on using it with PyTorch and Deep Learning. I origonally purchased it for how much memory it had 24GB. However, then I realized it has 2 gpus on a single board and the memory was split in half amongst them.

My question is: to train a neural network with the K80, would I have to load a copy of the Network into each of the 2 gpus on a single board for the K80? Or will just one copy do, and the other gpu on the chip can access it?

It also looks like I need to downgrade to pytorch 1.3. Is that correct?

Thank you for your help.

Could you please help with this K80? I’m not so sure it is usable for Deep Learning now a days.