What are the merits of partitioning shared memory into banks?

This forum, books about CUDA programming, and StackOverflow are full of questions about shared memory bank conflicts.

I am trying to understand the design backgrounds of shared memory banks and the merits of partitioning shared memory into banks. There must be some advantage to doing that, or it must be unavoidable/costly to avoid.

  • What are the advantages of shared memory banks?
  • Do other architectures (for example, L3 caches of CPUs) also have such banks? Could cache line contention be a good analogy for the this?

References to research papers, classes, or talks are appreciated.

I presume it has to do with memory hardware design. There is too much analogy to a 2D arrangement of cells even in the programmer’s model to ignore.

I imagine a 2D array of storage cells, where all cells in a column have a shared resource such as a shared data bus. I imagine that the alternative, to be able to take any cell’s data and route it to any of the other warp lanes would be prohibitive in terms of hardware design/layout area/cost.

This is not unlike DRAM design, which has row/column structure, and requires separate enablement of row and column to access data.

I have no idea if this is accurate or what the actual design looks like.

Constructing memory devices as a banked structure primarily has the goal of improving throughput, and often has the secondary benefit of improving latency compared with a non-banked device of identical capacity. Banking can be used at any level of the memory hierarchy, from register files down to system memory.

Banking of memory devices has been around for a long time, and I would expect any textbook on computer architecture to cover it: Have you checked Hennessy & Patterson? To find specific recent literature, pull up Google Scholar and search for banked cache, for example. This will result in thousands of hits, so use additional search terms to it narrow down to whatever you are interested in.

Thanks @njuffa and @Robert_Crovella for your instructive and kind replies. They were very helpful indeed.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.