# cudaMemcpy into shared variables

**URL:** <https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309>\
**Category:** CUDA Programming and Performance\
**Created:** [September 22, 2009, 9:29am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309 "2009-09-22T09:29:02Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 22, 2009, 9:29am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/1 "2009-09-22T09:29:02Z")

</div>

Hello,

i have the following single kernel

**global** void my\_kernel (cuFloatComplex\* data\_d, float\* x\_d, float\* y\_d)  
{  
**shared** float x[BLOCK\_SIZE],y[BLOCK\_SIZE];  
int ind\_col =(blockIdx.x_BLOCK\_SIZE)+threadIdx.x;  
int ind\_row =(blockIdx.y_BLOCK\_SIZE)+threadIdx.y;  
int ind\_m=blockIdx.z;  
cudaMemcpy(x,(x\_d+ind\_m_BLOCK\_SIZE),BLOCK\_SIZE,cudaMemcpyDev  
iceToHost);  
cudaMemcpy(y,(y\_d+ind\_m_BLOCK\_SIZE),BLOCK\_SIZE,cudaMemcpyDev  
iceToHost);  
\_\_syncthreads();

}

the grid is a 3D grid of (N2/BLOCK\_SIZE,N1\_BLOCK\_SIZE, M/BLOCK\_SIZE) and the Block is BLOCK\_SIZExBLOCK\_SIZE sized.

My first aim is to copy a segment of BLOCK\_SIZE dimension of input vector x\_d and y\_d into the Shared variables x and y. I have some doubts about.

First, can I use effectively cudaMemcpy? Or it doesn’t work inside a kernel?  
If it works, accoridng to my code, each single thread will copy a BLOCK\_SIZE amout of float data into the shared variables and hence I will have BLOCK\_SIZExBLOCK\_size threads writing BLOCK\_SIZE float data. So each kernel overwrites useless the shared variables with the same data. Is it correct?

I need to use the all the BLOCK\_SIZE x and y data in each of the threads of the block to perform a sum. So each thread of the block performs a weigthed sum based on BLOCK\_SIZE element of x\_d and y\_d.

Then, my question is: “How can I get an efficient copy from global memory into shared memory of the two vectors?”.

Does anybody have a suggestion?

Thanks in advance,

Beetro

---

<div class="post-metadata">

**Author:** ![CaLu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@CaLu](https://forums.developer.nvidia.com/u/CaLu)\
**Post date:** [September 22, 2009, 9:40am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/2 "2009-09-22T09:40:26Z")

</div>

You cannot call CudaMemcpy inside a kernel…

---

<div class="post-metadata">

**Author:** ![mickey\_mouse](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mickey\_mouse](https://forums.developer.nvidia.com/u/mickey_mouse)\
**Post date:** [September 22, 2009, 10:00am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/3 "2009-09-22T10:00:54Z")

</div>

What about using coalescing? Each thread copy one float in shared memory, something like

```auto
x[threadIdx.x + threadIdx.y * blockDim.x + threadIdx.z * blockDim.x * blockDim.y] = x_d[same index]

```

I don’t remember if floats makes any trouble in coalescing, but if it works I think it’s the better way to copy some data from global to shared memory…

EDIT: look at page 82 of CUDA programming guide version 2.3 for coalescence

---

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 22, 2009, 10:42am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/4 "2009-09-22T10:42:04Z")

</div>

> [@](#):
>
> You cannot call CudaMemcpy inside a kernel…

Ok, many thanks. I guessed it was not allowed. By the way, the code is compiled even if there a cudaMemcpy call in a kernel.

---

<div class="post-metadata">

**Author:** ![mickey\_mouse](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mickey\_mouse](https://forums.developer.nvidia.com/u/mickey_mouse)\
**Post date:** [September 22, 2009, 10:47am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/5 "2009-09-22T10:47:40Z")

</div>

> [@](#):
>
> Ok, many thanks. I guessed it was not allowed. By the way, the code is compiled even if there a cudaMemcpy call in a kernel.

I think that is translated in a standard memcpy, but I don’t know how it’s executed in the kernel

---

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 22, 2009, 10:56am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/6 "2009-09-22T10:56:10Z")

</div>

> [@](#):
>
> What about using coalescing? Each thread copy one float in shared memory, something like
> 
> ```auto
> x[threadIdx.x + threadIdx.y * blockDim.x + threadIdx.z * blockDim.x * blockDim.y] = x_d[same index]
> 
> ```
> 
> I don’t remember if floats makes any trouble in coalescing, but if it works I think it’s the better way to copy some data from global to shared memory…
> 
> EDIT: look at page 82 of CUDA programming guide version 2.3 for coalescence

yes, I want to perform coalescent access to memory, but my task is this: I need to multiply a 3D matrix (MxN1xN2), say A, with a vector C (1xM). Hence I want to do R=C\*A. But, before evaluating R I need to generate A.

Solution I) I could use 3D grid of 3D blocks of threads that accomplish A filling with coalescent access to memory and then I can perform the multiplication with another kernel by exploiting shared memory in a fashion similar to the CUDA programming guide 2.3 section 3.2.2.

Solution II) As some operation in A generation and C\*A multiplicaiton are similar (memory accesses) I would like to join togheter the two operations in one kernel, minimizing the memory access to global memory. In this case, I should size grid of blocks and blocks such that the number of total threads is not equal to MxN1xN2 (the 3D A matrix overall size), but is equal to N1xN2 (R size). Hence, i have available a less number of threads each managing a piece of A.

I do not know what is the best choiche. Should I have to to implement both and understand the most effective one by means of experimental trials?

What do you think?

Thank you,

Pietro

---

<div class="post-metadata">

**Author:** ![CaLu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@CaLu](https://forums.developer.nvidia.com/u/CaLu)\
**Post date:** [September 22, 2009, 11:27am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/7 "2009-09-22T11:27:35Z")

</div>

> [@](#):
>
> Solution I) I could use 3D grid of 3D blocks of threads that accomplish A filling with coalescent access to memory and then I can perform the multiplication with another kernel by exploiting shared memory in a fashion similar to the CUDA programming guide 2.3 section 3.2.2.

Keep attention of max z-dimension of 3D blocks and grids!

In my experience what you have absolutely to avoid is a large amount of global memory accesses!

Nice to see some guys from Italian Universities! :-)

Ciao,

luca

---

<div class="post-metadata">

**Author:** ![mickey\_mouse](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mickey\_mouse](https://forums.developer.nvidia.com/u/mickey_mouse)\
**Post date:** [September 22, 2009, 11:31am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/8 "2009-09-22T11:31:49Z")

</div>

> [@](#):
>
> yes, I want to perform coalescent access to memory, but my task is this: I need to multiply a 3D matrix (MxN1xN2), say A, with a vector C (1xM). Hence I want to do R=C\*A. But, before evaluating R I need to generate A.
> 
> Solution I) I could use 3D grid of 3D blocks of threads that accomplish A filling with coalescent access to memory and then I can perform the multiplication with another kernel by exploiting shared memory in a fashion similar to the CUDA programming guide 2.3 section 3.2.2.
> 
> Solution II) As some operation in A generation and C\*A multiplicaiton are similar (memory accesses) I would like to join togheter the two operations in one kernel, minimizing the memory access to global memory. In this case, I should size grid of blocks and blocks such that the number of total threads is not equal to MxN1xN2 (the 3D A matrix overall size), but is equal to N1xN2 (R size). Hence, i have available a less number of threads each managing a piece of A.

Ok, I think solution II is better.

What about something like this:

```auto
for (unsigned int i = 0; i < M; i++) {

   // load in shared memory A[i][threadIdx.y * blockDim.x][threadIdx.x]

}

```

Maybe I’ve not well understood your problem…

But this way you have M coalesced read from global memory, isn’t it?

Michele

---

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 23, 2009, 8:51am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/9 "2009-09-23T08:51:20Z")

</div>

> [@](#):
>
> Keep attention of max z-dimension of 3D blocks and grids!
> 
> In my experience what you have absolutely to avoid is a large amount of global memory accesses!
> 
> Nice to see some guys from Italian Universities! :-)
> 
> Ciao,
> 
> luca

Grazie Luca for your advices.

Indeed, it seems that 3d grid are not allowed, differntly from blocks of threads that can be 3d-indexed. So I’m wondering about the reason of that choice. I mean, if I have a 3D structure I probably would exploit 3d-blocks and 3d-grids of blocks to cover all the structure, don’t you think?

Ciao,

Pietro

---

<div class="post-metadata">

**Author:** ![CaLu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@CaLu](https://forums.developer.nvidia.com/u/CaLu)\
**Post date:** [September 23, 2009, 9:00am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/10 "2009-09-23T09:00:04Z")

</div>

Ciao Pietro,

yes, a grid can only be 2D. I think that 3D blocks are enough to describe a 3D problem… What’s the field you’re working in? Mathematics?

Ciao,

luca

---

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 23, 2009, 9:01am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/11 "2009-09-23T09:01:28Z")

</div>

> [@](#):
>
> Ok, I think solution II is better.
> 
> What about something like this:
> 
> ```auto
> for (unsigned int i = 0; i < M; i++) {
> 
> // load in shared memory A[i][threadIdx.y * blockDim.x][threadIdx.x]
> 
> }
> 
> ```
> 
> Maybe I’ve not well understood your problem…
> 
> But this way you have M coalesced read from global memory, isn’t it?
> 
> Michele

I guess that with your example I can get coalescent acces to memory. Unfortunately, the values of M involved i my typical problem can be very big and i do not think that i can load all the M-values in the shared memory of a thread block. Indeed, I manage M value of even 10000 of order of magnitude, and as the type is a cuFLoatComplex, I need 40000 bytes =40 KB, that is \> 16 KB available with shared. So this strategy should be very effective (theoretically) up to 4096 elements, but over this threshold it should show a significative drop of performance.

I not completely sure of my reasoning as I’m not expert as I have been using cuda for few weeks. So, please, your and somebodyelse’s comments are will be really appreciated.

Thanks,

Pietro

---

<div class="post-metadata">

**Author:** ![Beetro](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Beetro](https://forums.developer.nvidia.com/u/Beetro)\
**Post date:** [September 23, 2009, 9:08am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/12 "2009-09-23T09:08:27Z")

</div>

> [@](#):
>
> Ciao Pietro,
> 
> yes, a grid can only be 2D. I think that 3D blocks are enough to describe a 3D problem… What’s the field you’re working in? Mathematics?
> 
> Ciao,
> 
> luca

Electromagnetics…and particularly Antenna synthesis and analysis.

By the way, in I want to exploit shared memory whitout a 3D grid, when dimensions of data structures are large, you cannot segment the 3rd dim (the depth for instance). This is exactly the problem I have. I want to generate a 3D matrix of values of a function F(u,v,w) on points corresponding to the cartesian product of three vector u (N2 elements) v(N1 elements) and w(M elements)…M, N1 and N2 present very large values and I cannot load oin Shared mem all the corresponding vector if I use a 2d grid of blocks…if I could have used 3D-grid, I would have loaded only a segment of the vector fitting in the shared mem.

Do you have any suggestion to workaround this?

Thanks.

---

<div class="post-metadata">

**Author:** ![CaLu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@CaLu](https://forums.developer.nvidia.com/u/CaLu)\
**Post date:** [September 23, 2009, 9:11am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-into-shared-variables/12309/13 "2009-09-23T09:11:05Z")

</div>

If you’re dealing with huge numbers, have a look at this link:  
[url=“[http://http.developer.nvidia.com/GPUGems3/gpugems3\_ch31.html](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch31.html)”][http://http.developer.nvidia.com/GPUGems3/gpugems3\_ch31.html[/url]](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch31.html%5B/url%5D)

Here data is stored on shared memory in steps.
