Synchronize Blocks Within CUDA kernel Your ipinion

Accelerated Computing CUDA CUDA Programming and Performance
1

Will NVIDIA be implementing a new method to synchronize all the blocks executing a kernel? Is it even possible with current hardware?

2

No and no.

3

The blocks belong to following three states at any instant,

  1. done, or

  2. execute on some SM, or

  3. wait because of no resources.

You cannot synchronize for ALL blocks. More precisely, you cannot synchronize for blocks of state 3.

But you can synchronize blocks of state 1 and 2.

4

I have an algorithm that requires read and write to global memory in an iterative process and need to sync blocks between each iteration. What would be the best way to do this? might __threadfence()?

Currently, I call the kernel from the host in each iteration:

for(…)
Cuda-kernel;

thanks in advance

5

Your method is the best method. You don’t want to use __threadfence().

6

Isnt there any overhead in this way or?