My program in cuda sometimes runs successfully but sometimes runs with segmentation fault. Even when I reduce my thread numbers to one, it may run with segmentation fault again. But when I increase the number of threads, the rate of segmentation faults in multiple runs grow.
I’m using Ubuntu as my operating system.
What could the problem be?
It’s hard to say what is going on, but if you are segfaulting, you’ll first want to debug on the CPU:
Compile your code in debug mode (-g) and run in gdb so you can see which line is segfaulting.
Run your code with valgrind to catch memory access errors on the CPU. The segfault might be caused by an earlier problem.
If it looks like your segfault is caused by bad results from CUDA, then you should:
Make sure you check the return codes from all CUDA functions. If they are not cudaSuccess, print out the error and quit immediately. If your segfault is caused by incorrectly using data from a failed kernel, this might catch that.
Run your program with cuda-memcheck to catch device memory accesses that are out-of-bounds, which can also lead to bad results.