Scheduler for GPU's

Hello everyone.
We are a small team of researchers, we have a server with 4 nvidia GPU’s(NVIDIA Tesla P100).
And we can not evenly distribute the consumption of the possibilities of the GPU.
Is there any analogue of the process scheduler but for the GPU?
Thank you in advance!

We work with tensorflow,caffe and others Deep learning tools.