# cudaMemcpy error "invalid argument" from in-kernel malloc'ed device mem buffer on cuda 4

**URL:** <https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751>\
**Category:** CUDA Programming and Performance\
**Created:** [February 10, 2012, 5:02am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751 "2012-02-10T05:02:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![fji](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@fji](https://forums.developer.nvidia.com/u/fji)\
**Post date:** [February 10, 2012, 5:02am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/1 "2012-02-10T05:02:23Z")

</div>

Hi,

I use to have a piece of code work well on CUDA 4.0. But after using CUDA 4.1 official release, I found cudaMemcpy from a device memory buffer, which is allocated within a kernel using dynamic memory, back to host memory buffer returns an “invalid argument” error.

I wrote a simple example program to show this problem as in the following: (also attached)

```auto
#define SIZE 1024

#define BYTE_C 87

#include <stdlib.h>

#include <stdio.h>

__global__ void alloc_kernel(char **buf_ptr, size_t size, char content) {

    if(threadIdx.x == 0 && blockIdx.x == 0) {

        *buf_ptr = (char *) malloc(size);

        for(int i = 0; i < size; i++) {

            (*buf_ptr)[i] = content;

        }   

    }   

}

#define CUDA_CALL(cmd) do { \

    cudaError_t err; \

    err = cmd; \

    if( err != cudaSuccess ) { \

        printf("%s line %d: CUDA Error: %s\n", __FILE__ , __LINE__ , cudaGetErrorString(err)); \

    } \

}while(0)

int main() {

    char **d_bufptr, *h_bufptr, *h_buf;

CUDA_CALL( cudaSetDevice(0) );

    CUDA_CALL( cudaMalloc((void **) &d_bufptr, sizeof(char *) ) );

alloc_kernel<<<1, 1>>>(d_bufptr, SIZE, BYTE_C);

CUDA_CALL( cudaDeviceSynchronize() );

CUDA_CALL( cudaMemcpy(&h_bufptr, d_bufptr, sizeof(char *), cudaMemcpyDeviceToHost) );

    h_buf = (char *)malloc(SIZE);

    CUDA_CALL( cudaMemcpy(h_buf, h_bufptr, SIZE, cudaMemcpyDeviceToHost) );

return 0;

}

```

In the line where I do cudaMemcpy near the end, that is the place this error is returned. The same error returns for page-locked host buffer.

Can anyone give me a hint on where I did anything wrong? Or is this a bug in the latest driver/CUDA release? Thanks!

My system: Intel Xeon E5507, GTX 480. CUDA 4.1 toolkit. Ubuntu 11.04. Kernel version 2.6.38-13-generic. Driver NVIDIA-Linux-x86\_64-285.05.33. This problem is also seen in another CentOS 5.5 system (Intel Xeon E5560, M2070. CUDA 4.1, kernel 2.6.18-194.el5.perfctr).  
[simple\_memcpy\_test.cu](https://forums.developer.nvidia.com/uploads/short-url/8Y2FxKZNzjR2OIQoaiUNkZgfE3H.cu) (972 Bytes)

---

<div class="post-metadata">

**Author:** ![fji](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@fji](https://forums.developer.nvidia.com/u/fji)\
**Post date:** [February 17, 2012, 4:41pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/2 "2012-02-17T16:41:15Z")

</div>

Hi, does someone verify my code’s problem?

To give more information on this, this code works on cuda 4.0 (runtime + driver), but not with cuda 4.1 (runtime + driver).  
Even if I use cuda 4.0 runtime with a 4.1’s driver, it still returns the same error. Does this mean there is driver bug somewhere in there?

Thanks!

---

<div class="post-metadata">

**Author:** ![L\_F](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@L\_F](https://forums.developer.nvidia.com/u/L_F)\
**Post date:** [February 17, 2012, 8:30pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/3 "2012-02-17T20:30:04Z")

</div>

Check syntax of cudaMemcpy, you are using h\_bufptr in this code the different ways - as an address of a pointer and as a pointer.

---

<div class="post-metadata">

**Author:** ![fji](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@fji](https://forums.developer.nvidia.com/u/fji)\
**Post date:** [February 19, 2012, 6:10pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/4 "2012-02-19T18:10:15Z")

</div>

Hi, Thanks, but I don’t get what you mean.

h\_bufptr, defined as a char pointer, is a host variable, whose value is a device memory address.

Therefore, in

CUDA\_CALL( cudaMemcpy(&h\_bufptr, d\_bufptr, sizeof(char \*), cudaMemcpyDeviceToHost) );

The device buffer address stored in d\_bufptr is transferred into h\_bufptr. After that, this device buffer address can be used on the host.

Then, in

CUDA\_CALL( cudaMemcpy(h\_buf, h\_bufptr, SIZE, cudaMemcpyDeviceToHost) );

The device buffer address stored in h\_bufptr is used to transfer the in-kernel malloc’ed device buffer to the host side into h\_buf.

Could you explain how this contradicts the syntax of cudaMalloc? Thanks!

> [@](#):
>
> Check syntax of cudaMemcpy, you are using h\_bufptr in this code the different ways - as an address of a pointer and as a pointer.

---

<div class="post-metadata">

**Author:** ![L\_F](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@L\_F](https://forums.developer.nvidia.com/u/L_F)\
**Post date:** [February 22, 2012, 1:12am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/5 "2012-02-22T01:12:50Z")

</div>

Sorry, didn’t check your code thoroughly. It can be a mix of host/device pointers along with explicitly defined direction for cudaMemcpy. Have you tried to allocate h\_buf via cudaHostAlloc and copy with default direction?

---

<div class="post-metadata">

**Author:** ![fji](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@fji](https://forums.developer.nvidia.com/u/fji)\
**Post date:** [February 23, 2012, 7:11pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/6 "2012-02-23T19:11:44Z")

</div>

Nope.  
It returns the same error message whether the host buffer is allocated through malloc, cudaMalloc, or cudaHostAlloc (with all different flags).  
And it is the same case for cudaMemcpy with cudaMemcpyDefault parameter. (I guess the error message will be a different one, perhaps Invalid Direction if this direction parameter is the problem).

Do you have a CUDA 4.1 environment to verify this bug? Thanks!

---

<div class="post-metadata">

**Author:** ![L\_F](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@L\_F](https://forums.developer.nvidia.com/u/L_F)\
**Post date:** [February 24, 2012, 9:18pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/7 "2012-02-24T21:18:15Z")

</div>

Yes, probably there is some bug in CUDA driver, I checked with run-time API (“Invalid argument”) and with driver API (“ **global** function call is not configured”). But as a workaround you may use cudaHostAlloc for h\_buf and then “memcpy” inside a kernel to copy from \*d\_bufptr to h\_buf, it works correctly.

---

<div class="post-metadata">

**Author:** ![fji](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@fji](https://forums.developer.nvidia.com/u/fji)\
**Post date:** [February 28, 2012, 2:07am UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/8 "2012-02-28T02:07:21Z")

</div>

I never know there is a “memcpy” function that can be called in a kernel.

Could you point to the relevant sections in the CUDA programming guide about this “memcpy” function?

> [@](#):
>
> Yes, probably there is some bug in CUDA driver, I checked with run-time API (“Invalid argument”) and with driver API (“ **global** function call is not configured”). But as a workaround you may use cudaHostAlloc for h\_buf and then “memcpy” inside a kernel to copy from \*d\_bufptr to h\_buf, it works correctly.

---

<div class="post-metadata">

**Author:** ![L\_F](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@L\_F](https://forums.developer.nvidia.com/u/L_F)\
**Post date:** [February 28, 2012, 3:20pm UTC](https://forums.developer.nvidia.com/t/cudamemcpy-error-invalid-argument-from-in-kernel-malloced-device-mem-buffer-on-cuda-4/25751/9 "2012-02-28T15:20:54Z")

</div>

There is no mention in “CUDA C Programming Guide” about memcpy (probably because it’s not an extention), but compiler handles it as well as other usual C-functions (e.g. memset).