Thanks for you reply.
Here are the system information.
| Device | Driver on host | cuda-compat | version of cuda-gdb | inlined info | nvcc |
|---|---|---|---|---|---|
| P100 | 450.80.02 | cuda-compat-12-1-530.30.02-1 | 12.1 | full | Built on Mon_Apr__3_17:16:06_PDT_2023 Cuda compilation tools, release 12.1,V12.1.105 Build cuda_12.1.r12.1/compiler.32688072_0 |
| H800 | 535.129.03 | cuda-compat-12-5-555.42.02-1 | 12.5 | truncated | Built on Wed_Apr_17_19:19:55_PDT_2024 Cuda compilation tools, release 12.5, V12.5.40 Build cuda_12.5.r12.5/compiler.34177558_0 |
And both nccl were built on running container with matched version of nvcc.
My cuda-gdb on cuda12.5 is build on my own. cuda-gdb segv