I faced the following problem. My algorithm has two parts, that are operating on the same data: the first part is running on CPU (no chance to parallelize and its sequential implementation on GPU is very very slow), the second part is running on GPU and it’s very fast. But since the second part needs results from the first part I have to copy data from the host to the device using cudaMemcpy. And these copies slow down the whole algorithm significantly. Of course I’m using pinned memory for the data that is being transfered to the device. But are there some other tricks how copies CPU<-> GPU can be accelerated?
The second problem is the consequence of the first one. In order to have benefit of using GPU I’m using 16 bits integer instead of 32bits (than I have less data to transfer). But using 16 bits brings another limit to my algorithm. (since values can be higher than 16 535). 24 bits integer would solve my problems (because as I said with 32bits I have to copy a lot), but this format doesn’t exist. Or maybe someone has an experience of using 24 bits integers on GPU?
Yep, that’s what I mean. I’ve seen mentioned here that H2D copy and kernel execution can only overlap on linear transfers using cudaMemcpy, not when copying to device arrays or pitch linear memory, although I haven’t found any mention of this in the programming guide.
The problem is that only 1/4 of the Labels array is processed in the right way. If I will run the code above only for one stream stream then output results will be correct. What can be the problem? Maybe I’m using streams wrongly? I’m using CUDA 2.1.
Recently I installed CUDA 2.3 and tried new features like write-combining memory and mapped memory. I’m using GTX 295. Using mapped memory doesn’t help in my case, my program became even slower. I think it’s just interface for programming, because anyway data has to be transferred to device, isn’t it?
The fastest way to copy data to device is using pinned memory on the host with “cudaHostAllocWriteCombined” property. Should it be really like that?