atomicAdd: to shared memory / to global memory=====which is faster? (for Turing or later)

[202476410arsmart]

Hi! I have went through different type of atomicAdd and now my algorithm is stucked by this, so I am wondering what is the latest version for the latest strucutre? Such as Turing, Amphere or Volta?

The comparisons are:

1. Firstly atomicAdd to shared memory then normally write to global memory

2. Directly write to global memory

I noticed a lot of discussions in this topic, till Maxwell architecture…And the conclusion till then is: must use int32 can shared AtomicAdd faster than global. Not sure whether anything changed in later structure. (maybe not, see no info in volta/turing/amphere’s tuning guide…)

Thank you!!!

==========================================
Some puzzles are: I see in a early official post says to shared memory is faster, but one post report it is actually slow. And those are quite old…

Posts report shared is slower:

Shared memory atomicAdd is slower than that of using global memory

Hi, I am testing the shared memory atomicAdd posted in CUDA Pro Tip: Optimized Filtering with Warp-Aggregated Atomics. I found the latency using shared memory atomicAdd was even 15%~25% slower than using global memory. The entire test code was copied from the blog above. Is there any possible reasons? My test platform is Jetson Xavier, with ubuntu 16.04 and cuda 10.2.

AtomicAdd in Shared memory is measured slower than in Global memory. Timing, Shared memory, Atomic o

Dear all, I wrote 2 kernels to see how much atomicAdd() to shared memory is faster than global memory. Kernel is simple: Just keep add from i=0 … ITER-1, under 16 threads in a block across 256/16 blocks. The result I cannot understand is: atomicAdd to Shared memory - 140ms atomicAdd to Direct to Global memory- 90ms It would be so appreciated if you drop a line. SK. Here’s simple codes: #define WARP_WIDTH 16 #define W 256 #define ITER 1000000 ///////////////AtomicAdd to Shared memory …

Official tip claims shared atomicAdd is faster

CUDA Pro Tip: Optimized Filtering with Warp-Aggregated Atomics | NVIDIA...