CUDA simple questions. please answer!

Accelerated Computing CUDA CUDA Programming and Performance
1

Hello,

suppose, I declared a call cinfig like <<<16, 16>>>, then

  1. Total 16*16=256 thredas will creat in a kernal function?

  2. The execution of blocks is sequential or random?

    I mean, after executing block0 , immediately  "block1" executed or "some other block" is executed?

  3. The execution of threads in ablock is sequential or random?

    I mean, after executing threda0 in a block , immediately “thread1 of same block” is executed or “some other thread of some other block” is executed?

  4. please answer above questions.

2

1- 16 threads for each of the 16 blocks will be created, so a total of 256
2- the execution of the blocs is parallelized as much as possible, so no order is guatanteed
3- the execution of the threads is parallelized as much as possible, so no order is guatanteed

3

  1. true

  2. Execution of blocks is random(sequential is not guaranteed).

  3. threads in a block executed concurrently(that’s the key), more precisely they collected into warps(32 thread) and then executed concurrently.

  4. I suggest you to carefully read Programming Guide.

4

@jesusgumbau and @MKo,

so we can say that there is no guarantee to serial execution for both blocks and threads in a block.

thanks

5

Exactly, however you can synchronize threads in a block by using __syncthreads()