Async copy

[eyalhir74]

Hi,
Is there a way to do an async memcopy on a NON-pinned memory?
I want to launch a kernel (which calculates some formulas) and while its running copy the data needed
for the next kernel. Problem is that it seems that i must use pinned memory and I can’t - I use textures
and in current CUDA version textures can’t be bounded to pinned memory.

thanks
eyal

[Tigga]

I’m fairly sure the answer is no.

I am suprised to find out that you can’t use textures. I was under the impression that memory on the device was memory on the device, and it didn’t matter how it was transferred. Thought pinned was just to do with host memory. That said, it’s been a while since I used streams…

[eyalhir74]

I’m fairly sure the answer is no.

This is pitty :( It could hide my PCI overhead…

I am suprised to find out that you can’t use textures. I was under the impression that memory on the device was memory on the device, and it didn’t matter how it was transferred. Thought pinned was just to do with host memory. That said, it’s been a while since I used streams…

Yup. I’ve opened a bug (583022) and it was fixed for future versions :)

The following items have been modified for this Bug:

• Bug disposition changed from “Bug - Fixed” to “Bug - Fixed”

• Bug action changed from “QA - Open - Verify to close” to “QA - Open - Verify to close”

• Customer Status changed from “Open” to “Open”

• A new Comment has been added

---

Bug Information

---

Customer Bug ID:

NVIDIA Bug ID: 583022

        Date: 8/4/2009 4:41:42 AM

Company/Division: GPU Computing

    Severity: High

    Priority: 1 - High

    Synopsis: Texture over pinned memory fails.

 Description: Hi,

I get “Invalid device pointer” when trying to bind a texture to a device pointer created with cudaHostAllocMapped and cudaHostAlloc (pinned memory pointer).

Is there a way to do this?

Reproducing code:

include <stdlib.h>

include <stdio.h>

include <string.h>

include <math.h>

// includes, project

include <cutil_inline.h>

include <cuda.h>

int main(int argc, char** argv)

{

unsigned int flags = cudaHostAllocMapped;

float *fData_d = NULL;

unsigned int iSamples = 2001;

unsigned int iSize = 2 * 189676; 

iSize *= iSamples;		// iSize = 558,983,352

cudaSetDevice(1);

cudaSetDeviceFlags(cudaDeviceMapHost);

printf( "Allocating [%u] bytes...\n", iSize * sizeof( float ) );

cudaHostAlloc((void **)&fData_h, iSize * sizeof( float ), flags );

                 cudaThreadSynchronize();

cudaHostGetDevicePointer((void **)&fData_d, (void *)fData_h, 0);

                 cudaThreadSynchronize();

cudaBindTexture( 0, tex_LargeFloat2, fData_d, iSize );   // This fails with invalide device pointer [17]

                 cudaThreadSynchronize();

cudaFreeHost( fData_d ); }

Compilation done with CUDA 2.3 beta:

1>“C:\CUDA\bin\nvcc.exe” -arch sm_13 -ccbin “C:\Program Files (x86)\Microsoft Visual Studio 8\VC\bin” -Xcompiler “/EHsc /W3 /nologo /O2 /Zi /MT " -I”“C:\CUDA\include”" -I"…/…/common/inc" -maxrregcount=32 --compile -o “test.cu.obj” “test.cu”

---

Latest Comment update from NVIDIA (8/21/2009 10:43:28 AM):

Eyal, a fix for this bug is in place and will be available with a future release.

[MisterAnderson42]

Hi,

Is there a way to do an async memcopy on a NON-pinned memory?

I want to launch a kernel (which calculates some formulas) and while its running copy the data needed

for the next kernel. Problem is that it seems that i must use pinned memory and I can’t - I use textures

and in current CUDA version textures can’t be bounded to pinned memory.

thanks

eyal

There is a very simple reason why pinned memory is required for async transfers. Proof: Assume the contrary unpinned memory can be used. An async transfer is started and backgrounded. Then the OS needs to do memory management and shuffles virtual pages around in physical memory. Poof! The pages the aysnc transfer is copying no longer exist where they were at the start of the transfer and you are copying garbage data to the GPU.

[psr]

Yup. I’ve opened a bug (583022) and it was fixed for future versions

I’ve encountered pretty much the same bug trying to use cudaMemcpy/2D over mapped pinned memory. Hopefully they’ve fixed that as well.