I need to generate random numbers in my project.
First, I tried to use random.h in .cu file directly. Then I also tried to generate random numbers in .cpp(use cpu) and send them to the device(GPU). I find these two methods have two different results. Does random.h have different behavior on GPU?
You mean the file OptiX SDK <version>\SDK\cuda\random.h?
This is unrelated to the random number generator itself, that is a more general thing about floating point operations.
If calculations involve floating point operations (e.g. here a division inside the rnd() function, or esp. fused multiply add (FMA) compared to individual multiply and add instructions) then there can be differences in the least significant bits of the floating point results between processors because these calculations could be implemented at different internal precision.
That also happens regularly when using the CUDA nvcc use_fast_math option when compiling, which is highly recommend when performance matters as in raytracing device code!
If you want these to be consistent, you would need to implement integer based solutions.
Related post on random number generators here:
https://forums.developer.nvidia.com/t/how-to-generate-random-numbers-in-optim6-5/277420/2