When we put our code into the tensorflow docker image and run it, only one GPU is used. However on this GPU the Memory usage is near 100% (31344MiB/32475MiB) , but the GPU usage is only around 20-30%. When we run the CNN with the defect images the Memory is full to and the GPU usage is at 70-80% (also only one GPU used).
The question here is how to also leverage the running to the other GPU so that the load for GPU-volatile and Memory could be balance out. ?