concurrent memory access

Accelerated Computing CUDA CUDA Programming and Performance
1

Hello to all,

i don’t know exactly what happen if:

  • two threads try to read the same address in global memory
  • two threads try to write the same address in global memory
  • two threads of the same block try to read the same address in shared memory
  • two threads of the same block try to write the same address in shared memory

are the access serialized, or are they concurrent and then with unpredictable results?

thanks in advance

Francesco

2

All in the programming guide, but I’ll try to do it from memory as I’m on a really slow laptop with bad internet and the pdf will take years to load…

#1 Results in an uncoalesced access on CC <= 1.1. Works fine though.

#2 You’ll get a race condition. If the threads are in the same warp it’s quite undefined exactly what happens IIRC, but at least one is guarenteed to succeed. In different warps you’ll just get a straight serialised race condition.

#3 This is fun. If I remember correctly an address picked in an undefined way from the accesses of all the threads in the half warp becomes the broadcast word. If the two reads you’re doing happen to be this broadcast word, then it’s conflict free, otherwise you’ll get a bank conflict.

#4 Straight race condition. One of the writes will overwrite the other.

So basically, reading is predictable in value, though varies in speed, writing is unpredictable it value.