[Request] Add --sysroot flag to nvcc

Hi,

Would it be possible to add the “–sysroot” flag to nvcc, similar to the one gcc has?

Use case: when cross-compiling for a given target (e.g. Jetson, DDPX), we want to link against the exact same libraries that are found in the rootfs of the target hardware.

This can be accomplished in GCC by passing --sysroot=/mnt/targetfs/, which will prepend all the library and include paths with the path specified by the sysroot flag.

This is however not possible in NVCC. Why is this a problem? NVCC has baked in a set of library directories:

$ nvcc -v test.cu
#$ _SPACE_= 
#$ _CUDART_=cudart
#$ _HERE_=/usr/local/cuda/bin
#$ _THERE_=/usr/local/cuda/bin
#$ _TARGET_SIZE_=
#$ _TARGET_DIR_=
#$ _TARGET_DIR_=targets/x86_64-linux
#$ TOP=/usr/local/cuda/bin/..
#$ NVVMIR_LIBRARY_DIR=/usr/local/cuda/bin/../nvvm/libdevice
#$ LD_LIBRARY_PATH=/usr/local/cuda/bin/../lib:
#$ PATH=/usr/local/cuda/bin/../nvvm/bin:/usr/local/cuda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#$ INCLUDES="-I/usr/local/cuda/bin/../targets/x86_64-linux/include"  
#$ LIBRARIES=  "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib"

When cross-compiling, it will pick the aarch64-linux-gnu version. However, it will still use the libraries existing under the host’s /usr/local. It would be better to use the ones that are actually installed in the filesystem where the application is going to be deployed. Otherwise there’s risk that the host has old libraries that don’t match those installed in the target hardware.

Adding the --sysroot flag would make this possible. It would allow the “LIBRARIES” variable above to become:

-L/path/to/targetfs/usr/local/cuda/bin/../targets/aarch64-linux-gnu/lib

Perfectly matching what’s on the target filesystem and avoiding potential errors.

Of course one can add the above line manually to their build system and it will work, but it doesn’t feel right to have 2 -L lines in the same build recipe, potentially pointing to different libraries.

Thanks!

2 Likes