# Warp-Aggregate AtomicAdd

**URL:** <https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130>\
**Category:** CUDA Programming and Performance\
**Created:** [March 9, 2015, 8:50pm UTC](https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130 "2015-03-09T20:50:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![peterSteele123](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@peterSteele123](https://forums.developer.nvidia.com/u/peterSteele123)\
**Post date:** [March 9, 2015, 8:50pm UTC](https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130/1 "2015-03-09T20:50:51Z")

</div>

Hello all,

I am looking to use warp-aggregate operations on a simple global summation using CUDA. This is purely a learning exercise and I am not sure if the performance would be worth it but here it goes.

Can anyone point me to an example of a reduction using CUDA such that a reduction is employed with warp-level granularity? Therefore any atomicAdd would be used for a single thread in each warp rather than every thread in every warp. Can this be done?

Thank you.

---

<div class="post-metadata">

**Author:** ![tera](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tera](https://forums.developer.nvidia.com/u/tera)\
**Post date:** [March 9, 2015, 11:28pm UTC](https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130/2 "2015-03-09T23:28:03Z")

</div>

Yes - look e.g. at the second example (“Shuffle Warp Reduce”) at [this Parallel Forall blog entry](http://devblogs.nvidia.com/parallelforall/faster-parallel-reductions-kepler/).

You can also move the data via shared memory instead if your GPU does not support the shuffle intrinsics.

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [March 10, 2015, 3:16am UTC](https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130/3 "2015-03-10T03:16:23Z")

</div>

A pretty good introduction to basic parallel reduction techniques (not using atomics) is here:

[url][http://developer.download.nvidia.com/assets/cuda/files/reduction.pdf[/url]](http://developer.download.nvidia.com/assets/cuda/files/reduction.pdf%5B/url%5D)

(including shared memory warp level)

In addition, the programming guide discusses warp reductions using shfl instructions.

[url][http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#warp-examples-reduction[/url]](http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#warp-examples-reduction%5B/url%5D)

And there is the blog that tera pointed out.

---

<div class="post-metadata">

**Author:** ![peterSteele123](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@peterSteele123](https://forums.developer.nvidia.com/u/peterSteele123)\
**Post date:** [March 10, 2015, 3:23am UTC](https://forums.developer.nvidia.com/t/warp-aggregate-atomicadd/37130/4 "2015-03-10T03:23:29Z")

</div>

Thanks for the info. I will give the shfl instructions a try and only call atomics on a single thread per warp.