Migration capability shows as no

I have 4 x UCS B200M4 blade servers each with an NVIDIA TEsla M6 card installed. I can power on vGPU enabled VMs on all four of the hosts and they run fine, however I am only able to migrate between 2 of the 4 hosts. All are running ESXi 7.0 U3 and are running driver version 535.129.03. Running nvidia-smi vgpu -m on each host shows the 2 that I can migrate between with Migration capability:Yes, the other 2 show Migration capability:No. How can I change this so I can migrate between all 4 hosts?