Remote debugging: `CUDA GDB init file` is ignored

Hi there,

I am debugging remote CUDA applications, for which I compile the project locally and upload and execute the executable on a remote server. During debugging, all library files from the remote are downloaded to my local machine, which slows down the debugging process significantly. As the debugger output tells me, I can set sysroot to point to the library files locally. I would like to do that, but I haven’t found a way to set sysroot.

When I execute the same CUDA application on a local GPU (Debug as → NVIDIA CUDA GDB Debugger) , I can choose in the Debug configuration under the Debugger tab what file should be loaded as CUDA GDB init file (defaults to .cuda-gdbinit). When I add a .cuda-gdbinit file to my project folder, I can change sysroot.

This does not work when running as a remote application (Debug as → NVIDIA CUDA GDB Debugger (Remote)). It seems the CUDA GDB init file option is simply ignored.

This issue has been reported before but no working solution was suggested yet:

My setup

I am using Eclipse C/C++ IDE version 2020-12 (4.18.0) with Nsight Eclipse Plugin from CUDA Toolkit version 11.2.0.
OS locally is Ubuntu 20.04 LTS, remote has Ubuntu 18.04.5 LTS, both systems have CUDA toolkit version 11.2.0 installed.

And here are two example output from the Debugger console

I have a .cuda-gdbinit file in my project directory with the line

set sysroot /

By default, sysroot is set to target://. Here is the output from a local debug session where the .cuda-gdbinit file was successfully read and changed sysroot to /:

Coalescing of the CUDA commands output is off.
NVIDIA (R) CUDA Debugger
11.2 release
Portions Copyright (C) 2007-2020 NVIDIA Corporation
GNU gdb (GDB) 8.3.1
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Temporary breakpoint 5, main (argc=1, argv=0x7fffffffd848) at main.cu:21
21	{

show sysroot
The current system root is "/".

And here is the output from a remote debugging session, where all the library files are downloaded because the .cuda-gdbinit file was ignored and sysroot is unchanged (target://):

Coalescing of the CUDA commands output is off.
NVIDIA (R) CUDA Debugger
11.2 release
Portions Copyright (C) 2007-2020 NVIDIA Corporation
GNU gdb (GDB) 8.3.1
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading /lib64/ld-2.27.so from remote target...
Reading /lib64/.debug/ld-2.27.so from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib64/ld-2.27.so from remote target...
0x000015555532b090 in ?? () from target:/lib64/ld-linux-x86-64.so.2
Reading /usr/local/cuda/lib64/libcurand.so.10 from remote target...
Reading /lib/x86_64-linux-gnu/librt.so.1 from remote target...
Reading /lib/x86_64-linux-gnu/libpthread.so.0 from remote target...
Reading /lib/x86_64-linux-gnu/libdl.so.2 from remote target...
Reading /usr/lib/x86_64-linux-gnu/libstdc++.so.6 from remote target...
Reading /lib/x86_64-linux-gnu/libm.so.6 from remote target...
Reading /lib/x86_64-linux-gnu/libgcc_s.so.1 from remote target...
Reading /lib/x86_64-linux-gnu/libc.so.6 from remote target...
Reading /lib/x86_64-linux-gnu/librt-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/librt-2.27.so from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/librt-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/f36706eb2e6eee4046c4fdca2a19540b2f6113.debug from remote target...
Reading /lib/x86_64-linux-gnu/.debug/f36706eb2e6eee4046c4fdca2a19540b2f6113.debug from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/f36706eb2e6eee4046c4fdca2a19540b2f6113.debug from remote target...
Reading /lib/x86_64-linux-gnu/libdl-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/libdl-2.27.so from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/libdl-2.27.so from remote target...
Reading /usr/lib/x86_64-linux-gnu/119a44a99758114620c8e9d8e243d7094f77f6.debug from remote target...
Reading /usr/lib/x86_64-linux-gnu/.debug/119a44a99758114620c8e9d8e243d7094f77f6.debug from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//usr/lib/x86_64-linux-gnu/119a44a99758114620c8e9d8e243d7094f77f6.debug from remote target...
Reading /lib/x86_64-linux-gnu/libm-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/libm-2.27.so from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/libm-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/9f3ae11120ec7c483bc9295345d836f5c104f7.debug from remote target...
Reading /lib/x86_64-linux-gnu/.debug/9f3ae11120ec7c483bc9295345d836f5c104f7.debug from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/9f3ae11120ec7c483bc9295345d836f5c104f7.debug from remote target...
Reading /lib/x86_64-linux-gnu/libc-2.27.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/libc-2.27.so from remote target...
Reading /dvs/p4/build/sw/rel/gpgpu/toolkit/r11.2/bin/lib/debug//lib/x86_64-linux-gnu/libc-2.27.so from remote target...

Temporary breakpoint 5, main (argc=1, argv=0x7fffffffe7a8) at main.cu:21
21	{

show sysroot
The current system root is "target://".

Any help appreciated. Am I missing something or is this a bug? If so, I’ll file a bug report. But wanted to double check here first.