I am debugging a multi-GPU program which runs fine when only one GPU is engaged.
When multiple (2) GPU is enabled, I encounter below error in cuda-gdb. Once I am there I cannot switch focus, backtrace is short. Would you please give me some hint? Where are STORE4() and NO_REST_OF_K() defined? Does CUDA kernel 0 indicating the first kernel function registered from my program?
CUDA Exception: Device Illegal Address
The exception was triggered in device 1.
Program received signal CUDA_EXCEPTION_10, Device Illegal Address.
[Switching focus to CUDA kernel 0, grid 14, block (0,14,0), thread (0,0,0), device 1, sm 0, warp 0, lane 0]
0x00007f354b7648b0 in STORE4 ()
#0 0x00007f354b7648b0 in STORE4 ()
#1 0x00007f354b764490 in NO_REST_OF_K<<<(1,19,1),(16,16,1)>>> ()
(cuda-gdb) cuda block 1
Request cannot be satisfied. CUDA focus unchanged.