system process on gpu


I have 3 GPUs of which 2 for machine learning, with compute mode set as Exclusive Mode. I noticed that sometimes the system allocates some processes to all GPUs and when that happens, the GPU which has other process in the memory does not take Machine Learning tasks. As a result before running ML task I need to kill some processes to clear GPU memory.

My question is: Is there any way of preventing the linux (Ubuntu) from allocating processes to selected GPUs?