I programed the same algorithms for GPU and CPU. Nevertheless, there are almost 2% errors between them, like 2.9859 and 2.9662 in CPU and GPU respectively. Is this due to the size of memory copy, or others?
Is this due to the size of memory copy, or others?
Possible but most likely not. Typically divergent answers are caused by algorithmic differences, accumulation, or the precision of the mathematics used.
First thing to determine is if difference is acceptable for your algorithm. For example, if you are accumulation millions of single precision values, very small variations of round-off error can be magnified greatly.
If it is not acceptable, then you need to investigate and determine at what point does divergence occur and why. When I investigate these types of issues, I’ll use temp arrays to capture intermediate calculations, comparing the GPU and CPU results. Eventually, you can narrow down the area of code where divergence first occurs and hopefully get a better understanding why.