Is default kernel execution concurrent? Or we have to enable MPS?

Accelerated Computing CUDA CUDA Programming and Performance
1

If I launch kernels from different processes on the same device (for example A100), are kernels executed concurrently?
Or MPS is necessary to enable concurrent kernel execution?

Best
Max

2

No, they are not automatically run concurrently from separate processes. That is one of the key features of MPS. Resource requirements for observing concurrent kernel execution would also have to be met, of course. A kernel that by itself uses up available execution resources on a GPU should not be expected to run concurrently with another kernel.

3

Thank you for the answer.
Is there any way to run kernels concurrently (if resource is enough) without MPS?

Best
Max

4

From different processes? No.

5

You could enable MIG mode for A100.

6

If from the same process, any method to make it?

7

Right now I am working on MIG, I need CMK as the baseline.

8

Sure, the CUDA concurrent kernels sample code could be a starting point/demonstrator. Plus there are probably dozens of questions on various forums from people trying to witness concurrent kernels.

9

Thank you for the help. I will have a look on it.