# Dynamic memory allocation during kernel execution Is it posible?

**URL:** <https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574>\
**Category:** CUDA Programming and Performance\
**Created:** [July 2, 2009, 1:50pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574 "2009-07-02T13:50:15Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kuroi](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Kuroi](https://forums.developer.nvidia.com/u/Kuroi)\
**Post date:** [July 2, 2009, 1:50pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/1 "2009-07-02T13:50:15Z")

</div>

Hi everybody!!

I know that memory allocation for device memory, has to be dynamically done from host using the cudaMalloc() function and freed with cudaFree(), and I am in the understanding that CUDA doesn’t let you do this from the device (i.e. in kernels), the question is that if you guys know if somebody have work on this, you know, constructing a “kernelCudaMalloc()” or something like that [External Image](http://hqnveipbwb20/public/style_emoticons/<#EMO_DIR#>/ermm.gif "Image hosted on another site. Click to open in a new tab."). ?. that can be called from the device while running a kernel…

---

<div class="post-metadata">

**Author:** ![avidday](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/avidday/32/59219_2.png) [@avidday](https://forums.developer.nvidia.com/u/avidday)\
**Post date:** [July 2, 2009, 2:12pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/2 "2009-07-02T14:12:35Z")

</div>

CUDA doesn’t support memory allocation from inside kernels and I don’t think it can easily be done. I haven’t seen any evidence that any of the current CUDA capable GPUs actually has what might be described as an autonomous MMU, and as a result the GPU is reliant the host driver to do at least some of the memory management functions you need to be able to have a kernel call malloc() or similar.

---

<div class="post-metadata">

**Author:** ![daxue](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@daxue](https://forums.developer.nvidia.com/u/daxue)\
**Post date:** [February 15, 2011, 11:09am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/3 "2011-02-15T11:09:16Z")

</div>

Since now malloc() for kernel has been released (I found no literature declaring it only works for 2.x devices) and it seems Tesla C1060 (device 1.3) has a MMU (see [this page](http://www.realworldtech.com/page.cfm?ArticleID=RWT090808195242&p=10)), I wonder if there is any possibility for C1060 to dynamically allocate memory in kernel or device functions, although I didn’t find any literature supporting this, yet.

I have tried malloc(), when compiling by nvcc with no options, it gave error:  
calling a host function from a **device** / **global** function is not allowed

When compiling using option -arch=sm\_20 or -arch=sm\_21, no error occurred. However, the running output was wrong, which is predictable since the device is 1.3 not 2.x.

So it seems malloc() is only available for device 2.x (although I see no literature saying this). Is it because of hardware difference between devices 1.3 and 2.x? But someone told me there is no real difference, so dynamic allocation should be possible on 1.3 devices. Is he correct?

Just in case, below is my system settings.

OS: Ubuntu 10.04 (64-bit)  
CUDA device: Tesla C1060 (x2)  
Driver: 260.19.26  
Toolkit: 3.2.16  
SDK: 3.2.16

---

<div class="post-metadata">

**Author:** ![njuffa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@njuffa](https://forums.developer.nvidia.com/u/njuffa)\
**Post date:** [February 15, 2011, 12:04pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/4 "2011-02-15T12:04:24Z")

</div>

Device-side malloc() requires sm\_2x. Due to an oversight, this is not being pointed out by the CUDA 3.2 documentation. This documentation defect was brought to our attention a couple of months and should be fixed in the next release.

---

<div class="post-metadata">

**Author:** ![daxue](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@daxue](https://forums.developer.nvidia.com/u/daxue)\
**Post date:** [February 15, 2011, 2:25pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/5 "2011-02-15T14:25:24Z")

</div>

Thanks for reply. Anyone knows the answer regarding to possibility and/or hardware difference?

---

<div class="post-metadata">

**Author:** ![huangshuojch](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@huangshuojch](https://forums.developer.nvidia.com/u/huangshuojch)\
**Post date:** [February 16, 2011, 8:32am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/6 "2011-02-16T08:32:43Z")

</div>

So, if I want to use the device-side malloc(), do I need to add the _device_, or just use malloc in the kernel?

---

<div class="post-metadata">

**Author:** ![avidday](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/avidday/32/59219_2.png) [@avidday](https://forums.developer.nvidia.com/u/avidday)\
**Post date:** [February 16, 2011, 9:25am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/7 "2011-02-16T09:25:16Z")

</div>

Just include stdlib.h and call malloc. If the code generation options are correct, nvcc will overload the call with the device version of the function automagically.

As for hardware differences, it worth noting that my original reply in this thread was written 3 months before Fermi was announced. At the time of the announcement, NVIDIA made a big deal about redesigning the GPU memory controller to provide a unified “flat” address space that would allow device code malloc and the C++ new operator to work. This implies that GT200 memory controller wasn’t a full MMU in the conventional sense. The Fermi implementation of malloc still requires a host driver managed heap that must be sized and allocated before the kernel runs. That tells me that even in Fermi, the MMU isn’t completely autonomous and some kind of host driver managed memory map is still used.

---

<div class="post-metadata">

**Author:** ![svbettscheider](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@svbettscheider](https://forums.developer.nvidia.com/u/svbettscheider)\
**Post date:** [February 16, 2011, 9:33am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/8 "2011-02-16T09:33:06Z")

</div>

> [@](#):
>
> When compiling using option -arch=sm\_20 or -arch=sm\_21, no error occurred. However, the running output was wrong, which is predictable since the device is 1.3 not 2.x.

The same by me. No error occurs, by compiling my code. But the execution of the executable goes wrong.

---

<div class="post-metadata">

**Author:** ![daxue](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@daxue](https://forums.developer.nvidia.com/u/daxue)\
**Post date:** [February 18, 2011, 3:13am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/9 "2011-02-18T03:13:18Z")

</div>

> [@](#):
>
> But someone told me there is no real difference, so dynamic allocation should be possible on 1.3 devices. Is he correct?

How about this?

---

<div class="post-metadata">

**Author:** ![hardypete](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hardypete](https://forums.developer.nvidia.com/u/hardypete)\
**Post date:** [July 20, 2011, 8:55am UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/10 "2011-07-20T08:55:52Z")

</div>

> [@](#):
>
> The same by me. No error occurs, by compiling my code. But the execution of the executable goes wrong.

Hi. Has anyone found a solution to this? I have the same problem when running my code, compiles fine with a malloc inside the kernel, no errors, but causes unexpected behaviour at runtime, even when the kernel containing the malloc is not executed . . .

Ta, Pete

Btw, I’m running on a Tesla C2070 on 64bit ubuntu

---

<div class="post-metadata">

**Author:** ![cuuuuuda](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@cuuuuuda](https://forums.developer.nvidia.com/u/cuuuuuda)\
**Post date:** [October 24, 2011, 1:43pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/11 "2011-10-24T13:43:40Z")

</div>

I was able to malloc the data in kernel and transfer back to host and check it.

#include “stdlib.h”  
#define ALLOC\_SIZE 128  
**global** void  
test\_malloc(int \*\*controller)  
{  
**shared** int _ptr;  
int bx = blockIdx.x;  
if (threadIdx.x == 0)  
{  
ptr = (int_)malloc(ALLOC\_SIZE \* sizeof(int));  
controller[bx] = ptr;  
}  
\_\_syncthreads();  
for (int idx = threadIdx.x; idx \< ALLOC\_SIZE; idx += blockDim.x)  
{  
ptr[idx] = threadIdx.x;  
}  
}

**global** void  
test\_free(int \*\*controller)  
{  
int bx = blockIdx.x;  
if(threadIdx.x == 0)  
{  
free(controller[bx]);  
}  
}

void runTest()  
{  
int block\_num = 64;  
int block\_size = 32;  
int \*\*g\_controller;  
cudaMalloc(&g\_controller, sizeof(int \*) \*block\_num);  
test\_malloc\<\<\<block\_num, block\_size\>\>\>(g\_controller);  
int \*h\_controller[block\_num];  
cudaMemcpy(h\_controller, g\_controller, sizeof(int \*) \*block\_num, cudaMemcpyDeviceToHost);  
for (int i = 0; i != block\_size; i++)  
{  
printf(“allocated pointer %p.\n”, h\_controller[i]);  
int buffer[ALLOC\_SIZE];  
cudaMemcpy(buffer, h\_controller[i], sizeof(int) \* ALLOC\_SIZE, cudaMemcpyDeviceToHost);  
for (int index = 0; index != block\_size; index++)  
assert(buffer[index] == index);

```
    }
  test_free<<<block_num, block_size>>>(g_controller);

```

}

---

<div class="post-metadata">

**Author:** ![haque](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@haque](https://forums.developer.nvidia.com/u/haque)\
**Post date:** [January 25, 2013, 5:32pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/12 "2013-01-25T17:32:25Z")

</div>

I try to compile the above code. But it produce compile time errors:

test.cu(19): error: calling a host function from a **device** / **global** function is only allowed in device emulation mode

test.cu(51): error: calling a host function from a **device** / **global** function is only allowed in device emulation mode

Can you please help me running this code?

---

<div class="post-metadata">

**Author:** ![alrikai](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/alrikai/32/80956_2.png) [@alrikai](https://forums.developer.nvidia.com/u/alrikai)\
**Post date:** [January 25, 2013, 7:13pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/13 "2013-01-25T19:13:09Z")

</div>

> [@haque](#):
>
> I try to compile the above code. But it produce compile time errors:
> 
> test.cu(19): error: calling a host function from a **device** / **global** function is only allowed in device emulation mode
> 
> test.cu(51): error: calling a host function from a **device** / **global** function is only allowed in device emulation mode
> 
> Can you please help me running this code?

How did you compile it? Specifically, for what compute capability? It sounds like you were targetting one that’s too old, you need 2.x or more recent for dynamic allocation on the device

---

<div class="post-metadata">

**Author:** ![njuffa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@njuffa](https://forums.developer.nvidia.com/u/njuffa)\
**Post date:** [January 25, 2013, 7:24pm UTC](https://forums.developer.nvidia.com/t/dynamic-memory-allocation-during-kernel-execution-is-it-posible/10574/14 "2013-01-25T19:24:24Z")

</div>

Device-side malloc() requires compute capability 2.0 or higher (i.e. a Fermi or Kepler class GPU). BTW, you may want to update your CUDA version. The mention of device emulation in the quoted error message suggests you are using a CUDA version prior to 3.1. The current CUDA version is 5.0.
