Advice on porting to an HPC application to GPU

[MatColgrove]

I’m not certain on the details, but my understanding is that there’s not a limit on the number of kernels that can be launched, but there is a limit to the number of kernels on the launch queue. So once the queue fills up, the next kernel launch need to block waiting for a spot to open up.

I found this post which you might find helpful:

Maximum number of operations in a stream

Hello, I am currently programming an HPC application where I submit many operations (my own kernels, cublas/cusparse/cusolver routines) into a single stream. I ran into a problem that the submission starts being synchronous after a while. The kernel launch should be asynchronous, but I saw that it took “a lot of time” to launch the kernel. I boiled the problem down to a simple program (see below), where I found out that after I submit about 1022 kernels into a stream (no difference between a n…