How to run different executable files in parallel on GPU?

I am trying to run two different executable files in parallel on GPU. I am compiling the same code with different parameters. For 1 executable file, my GPU utilisation is 40 %. So, I am thinking to run two different executable file having 80% GPU utilisation giving me output in the same time. But, the time taken is double the 1 execution time.

I am running this command to run: ./a1.out & ./a2.out, time taken= 60 sec.
when I am giving this command only: ./a1.out, time taken = 30 sec only.

Please help me out here. Please tell me if there is a way to run files parallely on GPU.

If you simply run two files in 2 separate processes, the kernels will serialize. A GPU does not run kernels from separate processes concurrently, unless you use CUDA MPS. Even then, they may not run concurrently.

But it is showing my GPU utilisation double as compared to single one. So, what does make my GPU utilisation double if they are not running concurrently??

is it really shows that gpu utilization is 80%?

overall, therу is no common definition of “GPU utilization”. When nvprof shows that only 40% of ALU ticks are utilized, it’s usually because other GPU resources required by the kernel turned out to be more tight limiter than ALUs. Executing multiple programs simultaneously cannot improve this situation.

The things are different on CPUs. Here, if your CPU is utilized by 50%, this means that half of cores are idle, so you can run another program on these cores. On GPU, kernels are usually include many thousands of threads, so entire GPU is almost always utilized by the single kernel. Thus, it’s meaningless to show GPU utilization as number of cores involved into kernel computations - it will be almost always 100%, absolutely non-informative.

So, various GPU profiler tools show which percent of ALU cycles are utilized. If you will show similar stats for CPU, you will also see that it’s never 100% due to various delays. Unfortunately, this doesn’t mean that you can run more threads (on CPU or GPU), it just means that other scarce resources limit ALU utilization

I have installed nvdia-settings to see the percentage utilisation of GPU.This is the image of screen I am seeing for 1 executable file and similarly executing the files concurently shows me 80% utilisation.