Submitting multiple jobs to GPU at the same time

I wonder what will happen if multiple jobs are submitted to the GPU at the same time. Ppl in my lab work on the same remote machine, and sometimes when I’m running a job on the GPU (which has 4 units), my program takes 4 units, another person submits a job which also requires 4 GPU units. I wonder what will happen now? Will the GPU finish my job first, or it will try to run both jobs simultaneously? Thanks!

It depends on the Operating System and the work of load of the GPU. But, generally, the driver will try to execute them “simultaneously”.

Thank you for your reply! But will the processing speed of GPU decrease when ‘simultaneously processing multiple jobs’? Say both job 1 and job 2 takes 1 hour to process independently, will the processing time be 2 hours or much longer if job 1 and job 2 are submitted at the same time?

In this case it depends on the workload of the job, the occupancy of your kernels, copies, latency…
If a job is using 100% of the resources, then job1 + job2 will take some more than 2 hours. On the other hand, for instance, if job1 uses only 50%, probably, job1 + job2 will take more than 1 hour but less than 2.

Again, it depends on several factors, the best way to know it, is to test it.

Thank you so much!

This only works if you HyperQ which one can get only for card 3.5 and higher.

And HyperQ only works for, at best, threads within a process, as far as I know.

In practice if you have more than one program running on the same GPU the best you can do is that while 1 program does something on the cpu like IO operations the other will use the gpu to run their kernels.