Simplified way to copy data to

[Letharion]

I’ve written these templates to simplify things for me. Hopefully it can be of some use to someone else.

[codebox]template

T* SimpleCudaMalloc(size_t size) {

T* devicePtr = NULL;

cudaError_t error = cudaMalloc((void**)&devicePtr, size * sizeof(T));

if (error == cudaErrorMemoryAllocation) {

  cout << "CudaMalloc error: " << error << " (" << cudaGetErrorString(error) << ")" << ". Exiting" << endl;

  assert(false);

}

return devicePtr;

}

template

T* SimpleCudaMallocCopyHtD(T* hostPtr, size_t size) {

T* devicePtr = SimpleCudaMalloc(size);

cudaError_t error = cudaMemcpy(devicePtr, hostPtr, sizeof(T) * size, cudaMemcpyHostToDevice);

if(error != 0) {

  cout << "MallocCopy HtD error: " << error << " (" << cudaGetErrorString(error) << ")" << ". Exiting" << endl;

  assert(false);

}

return devicePtr;

}

template

void SimpleCudaCopyHtD(T* hostPtr, T* devicePtr, size_t size) {

cudaError_t error = cudaMemcpy(devicePtr, hostPtr, sizeof(T)*size, cudaMemcpyHostToDevice);

if(error != 0) {

  cout << "Copy HtD error: " << error << " (" << cudaGetErrorString(error) << ")" << ". Exiting" << endl;

  assert(false);

}

}

template

void SimpleCudaCopyDtH(T* hostPtr, T* devicePtr, size_t size) {

cudaError_t error = cudaMemcpy(hostPtr, devicePtr, sizeof(T)*size, cudaMemcpyDeviceToHost);

if(error != 0) {

  cout << "Copy DtT error: " << error << " (" << cudaGetErrorString(error) << ")" << ". Exiting" << endl;

  assert(false);

}

}

//They could be invoked like this

float* VarSource, VarDest;

int Size = numberOfElements;

VarDest = SimpleCudaMallocCopyHtD(VarSource, Size);

//or

VarDest = SimpleCudaMalloc(Size);[/codebox]

[Jamie_K]

If you are, like me, somewhat new to C, and/or even worse, pointers all together, Cuda might seem intimidating.

Personally, I’ve made loads and loads of mistakes over the last few days when trying to learn.

I’ve now written these two templates to simplify things for me. Ofc this sacrifices a lot of flexibility, but at least they suit me well.

I’m posting them here because

( A ) I’d like some feedback from anyone who knows Cuda better than me (And hopefully they can be improved, or I will be told why they are a bad idea all together and they should be abandoned)

( B ) I hope some else who’s a n00b can benefit from them.

…

//They could be invoked like this

float* VarA, VarAK, VarB;

SimpleCudaMallocCopyHtD(VarA, VarAK, Size);

SimpleCudaMalloc(VarB, Size);

Something you may want to double check, does SimpleCudaMalloc successfully modify it’s input parameter? Because it appears that it’s passing the pointer by value, modifying it, and returning the modified version, without actually modifying the original.

In other words, it looks like this would work:

VarB = SimpleCudaMalloc(VarB, Size);

but perhaps this would not:

SimpleCudaMalloc(VarB, Size);

Perhaps a simpler version would not require a pointer to be passed as input at all, say like so:

[codebox]

template

T* SimpleCudaMalloc(size_t size) {

T* devPtr;

int error = cudaMalloc((void**)&devPtr, size * sizeof(T));

if (error == cudaErrorMemoryAllocation) {

  printf("\n Memory allocation error on device, exiting\n");

  exit(1);

}

return devPtr;

}

template

T* SimpleCudaMallocCopyHtD(const T* source, size_t size) {

T *destinationPtr = SimpleCudaMalloc(destinationPtr, size);

int error = cudaMemcpy(destinationPtr, source, sizeof(T) * size, cudaMemcpyHostToDevice);

if(error != 0) {

  printf("\nMemory copy error, exiting\n");

  exit(1);

}

return destinationPtr;

}

[/codebox]

Another suggestion would be in the case of error to return a value indicating an error (NULL seems like a good candidate) or to throw an exception. Unceremoniously aborting the entire program with exit(1) is not my preference but if it suits you it’s not wrong.

[hdinh]

Thanks! I will definitely try this code out. I’ve previously read that CUDA have some limitations relating to classes. Have you had any trouble allocating class objects or even structs objects on device?

Something you may want to double check, does SimpleCudaMalloc successfully modify it’s input parameter? Because it appears that it’s passing the pointer by value, modifying it, and returning the modified version, without actually modifying the original.

In other words, it looks like this would work:

VarB = SimpleCudaMalloc(VarB, Size);

but perhaps this would not:

SimpleCudaMalloc(VarB, Size);

Perhaps a simpler version would not require a pointer to be passed as input at all, say like so:

[codebox]

template

T* SimpleCudaMalloc(size_t size) {

T* devPtr;

int error = cudaMalloc((void**)&devPtr, size * sizeof(T));

if (error == cudaErrorMemoryAllocation) {

  printf("\n Memory allocation error on device, exiting\n");

  exit(1);

}

return devPtr;

}

template

T* SimpleCudaMallocCopyHtD(const T* source, size_t size) {

T *destinationPtr = SimpleCudaMalloc(destinationPtr, size);

int error = cudaMemcpy(destinationPtr, source, sizeof(T) * size, cudaMemcpyHostToDevice);

if(error != 0) {

  printf("\nMemory copy error, exiting\n");

  exit(1);

}

return destinationPtr;

}

[/codebox]

Another suggestion would be in the case of error to return a value indicating an error (NULL seems like a good candidate) or to throw an exception. Unceremoniously aborting the entire program with exit(1) is not my preference but if it suits you it’s not wrong.

[Letharion]

"Something you may want to double check, does SimpleCudaMalloc successfully modify it’s input parameter? Because it appears that it’s passing the pointer by value, modifying it, and returning the modified version, without actually modifying the original.

In other words, it looks like this would work:

VarB = SimpleCudaMalloc(VarB, Size);

but perhaps this would not:

SimpleCudaMalloc(VarB, Size);

Perhaps a simpler version would not require a pointer to be passed as input at all, say like so:

[codebox]

template

T* SimpleCudaMalloc(size_t size) {

T* devPtr;

int error = cudaMalloc((void**)&devPtr, size * sizeof(T));

if (error == cudaErrorMemoryAllocation) {

  printf("\n Memory allocation error on device, exiting\n");

  exit(1);

}

return devPtr;

}

template

T* SimpleCudaMallocCopyHtD(const T* source, size_t size) {

T *destinationPtr = SimpleCudaMalloc(destinationPtr, size);

int error = cudaMemcpy(destinationPtr, source, sizeof(T) * size, cudaMemcpyHostToDevice);

if(error != 0) {

  printf("\nMemory copy error, exiting\n");

  exit(1);

}

return destinationPtr;

}

[/codebox]

Another suggestion would be in the case of error to return a value indicating an error (NULL seems like a good candidate) or to throw an exception. Unceremoniously aborting the entire program with exit(1) is not my preference but if it suits you it’s not wrong.

Your quite right about the return value, I was quite aware of it at the time, but obviously was to tired to mention it ;)

Yes, there could definitely be some better error handling. I’ll update the above later, probably with both your suggestions. Even if one does want to just exit(1), maybe that should be left up to each program(mer) and not be hardcoded.

“CUDA have some limitations relating to classes.”

Well, that’s a bit of an understatement considering that Cuda still AFAIK still only supports C, so classes as just not supported at all (On the device).

Haven’t tried allocating a struct however, but that shouldn’t be a problem.

[Letharion]

The templates have been somewhat simplified, thanks to Jamie.
Works well so far. Error handling could still use some improvment, but they current version works well for me, so if noone else cares, I’ll leave it as it is.

[Letharion]

Updated the templates somewhat.

They still die on failure, but that’s easily altered if it doesn’t suit you.