Nsys and RUNPATH loading

Developer Tools Nsight Systems Profiling Linux Targets
1

I’m trying to use Nsight Systems with Julia, but am hitting an issue where it attempts to open the wrong libpcre library. Julia sets its RUNPATH to be $ORIGIN/../lib:$ORIGIN/../lib/julia and the library is in the second of these directories. However when I run

LD_DEBUG=libs nsys launch julia

I get the following piece:

     21111:	find library=libpcre2-8 [0]; searching
     21111:	 search path=/central/software/nsight-systems/2023.2.1/target-linux-x64		(RPATH from file /central/software/nsight-systems/2023.2.1/target-linux-x64/libToolsInjectionProxy64.so)
     21111:	  trying file=/central/software/nsight-systems/2023.2.1/target-linux-x64/libpcre2-8
     21111:	 search path=/central/software/julia/1.8.5/lib:/central/slurm/install/current/lib:/central/software/hdf5/1.14.0-ompi415//lib:/central/software/OpenMPI/4.1.5//lib:/central/software/ucx/1.13.1_cuda-11.2//lib:/central/software/CUDA/11.2/lib64:/central/software/CUDA/11.2/extras/CUPTI/lib64:/central/software/CUDA/11.2/targets/x86_64-linux/lib		(LD_LIBRARY_PATH)
     21111:	  trying file=/central/software/julia/1.8.5/lib/libpcre2-8
     21111:	  trying file=/central/slurm/install/current/lib/libpcre2-8
     21111:	  trying file=/central/software/hdf5/1.14.0-ompi415//lib/libpcre2-8
     21111:	  trying file=/central/software/OpenMPI/4.1.5//lib/libpcre2-8
     21111:	  trying file=/central/software/ucx/1.13.1_cuda-11.2//lib/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/lib64/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/extras/CUPTI/lib64/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/targets/x86_64-linux/lib/libpcre2-8
     21111:	 search cache=/etc/ld.so.cache
     21111:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     21111:	  trying file=/lib64/tls/libpcre2-8
     21111:	  trying file=/lib64/libpcre2-8
     21111:	  trying file=/usr/lib64/tls/libpcre2-8
     21111:	  trying file=/usr/lib64/libpcre2-8
     21111:	
     21111:	find library=libpcre2-8 [0]; searching
     21111:	 search path=/central/software/nsight-systems/2023.2.1/target-linux-x64		(RPATH from file /central/software/nsight-systems/2023.2.1/target-linux-x64/libToolsInjectionProxy64.so)
     21111:	  trying file=/central/software/nsight-systems/2023.2.1/target-linux-x64/libpcre2-8
     21111:	 search path=/central/software/julia/1.8.5/lib:/central/slurm/install/current/lib:/central/software/hdf5/1.14.0-ompi415//lib:/central/software/OpenMPI/4.1.5//lib:/central/software/ucx/1.13.1_cuda-11.2//lib:/central/software/CUDA/11.2/lib64:/central/software/CUDA/11.2/extras/CUPTI/lib64:/central/software/CUDA/11.2/targets/x86_64-linux/lib		(LD_LIBRARY_PATH)
     21111:	  trying file=/central/software/julia/1.8.5/lib/libpcre2-8
     21111:	  trying file=/central/slurm/install/current/lib/libpcre2-8
     21111:	  trying file=/central/software/hdf5/1.14.0-ompi415//lib/libpcre2-8
     21111:	  trying file=/central/software/OpenMPI/4.1.5//lib/libpcre2-8
     21111:	  trying file=/central/software/ucx/1.13.1_cuda-11.2//lib/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/lib64/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/extras/CUPTI/lib64/libpcre2-8
     21111:	  trying file=/central/software/CUDA/11.2/targets/x86_64-linux/lib/libpcre2-8
     21111:	 search cache=/etc/ld.so.cache
     21111:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     21111:	  trying file=/lib64/tls/libpcre2-8
     21111:	  trying file=/lib64/libpcre2-8
     21111:	  trying file=/usr/lib64/tls/libpcre2-8
     21111:	  trying file=/usr/lib64/libpcre2-8
     21111:	
     21111:	find library=dlopen hook: 'libpcre2-8' [0]; searching
     21111:	 search path=/central/software/nsight-systems/2023.2.1/target-linux-x64		(RPATH from file /central/software/nsight-systems/2023.2.1/target-linux-x64/libToolsInjectionProxy64.so)
     21111:	  trying file=/central/software/nsight-systems/2023.2.1/target-linux-x64/dlopen hook: 'libpcre2-8'
     21111:	 search path=/central/software/julia/1.8.5/lib:/central/slurm/install/current/lib:/central/software/hdf5/1.14.0-ompi415//lib:/central/software/OpenMPI/4.1.5//lib:/central/software/ucx/1.13.1_cuda-11.2//lib:/central/software/CUDA/11.2/lib64:/central/software/CUDA/11.2/extras/CUPTI/lib64:/central/software/CUDA/11.2/targets/x86_64-linux/lib		(LD_LIBRARY_PATH)
     21111:	  trying file=/central/software/julia/1.8.5/lib/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/slurm/install/current/lib/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/hdf5/1.14.0-ompi415//lib/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/OpenMPI/4.1.5//lib/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/ucx/1.13.1_cuda-11.2//lib/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/CUDA/11.2/lib64/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/CUDA/11.2/extras/CUPTI/lib64/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/central/software/CUDA/11.2/targets/x86_64-linux/lib/dlopen hook: 'libpcre2-8'
     21111:	 search cache=/etc/ld.so.cache
     21111:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     21111:	  trying file=/lib64/tls/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/lib64/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/usr/lib64/tls/dlopen hook: 'libpcre2-8'
     21111:	  trying file=/usr/lib64/dlopen hook: 'libpcre2-8'
     21111:	
     21111:	find library=libpcre2-8.so [0]; searching
     21111:	 search path=/central/software/nsight-systems/2023.2.1/target-linux-x64		(RPATH from file /central/software/nsight-systems/2023.2.1/target-linux-x64/libToolsInjectionProxy64.so)
     21111:	  trying file=/central/software/nsight-systems/2023.2.1/target-linux-x64/libpcre2-8.so
     21111:	 search path=/central/software/julia/1.8.5/lib:/central/slurm/install/current/lib:/central/software/hdf5/1.14.0-ompi415//lib:/central/software/OpenMPI/4.1.5//lib:/central/software/ucx/1.13.1_cuda-11.2//lib:/central/software/CUDA/11.2/lib64:/central/software/CUDA/11.2/extras/CUPTI/lib64:/central/software/CUDA/11.2/targets/x86_64-linux/lib		(LD_LIBRARY_PATH)
     21111:	  trying file=/central/software/julia/1.8.5/lib/libpcre2-8.so
     21111:	  trying file=/central/slurm/install/current/lib/libpcre2-8.so
     21111:	  trying file=/central/software/hdf5/1.14.0-ompi415//lib/libpcre2-8.so
     21111:	  trying file=/central/software/OpenMPI/4.1.5//lib/libpcre2-8.so
     21111:	  trying file=/central/software/ucx/1.13.1_cuda-11.2//lib/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/lib64/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/extras/CUPTI/lib64/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/targets/x86_64-linux/lib/libpcre2-8.so
     21111:	 search cache=/etc/ld.so.cache
     21111:	  trying file=/lib64/libpcre2-8.so
     21111:	
     21111:	find library=libpcre2-8.so [0]; searching
     21111:	 search path=/central/software/nsight-systems/2023.2.1/target-linux-x64		(RPATH from file /central/software/nsight-systems/2023.2.1/target-linux-x64/libToolsInjectionProxy64.so)
     21111:	  trying file=/central/software/nsight-systems/2023.2.1/target-linux-x64/libpcre2-8.so
     21111:	 search path=/central/software/julia/1.8.5/lib:/central/slurm/install/current/lib:/central/software/hdf5/1.14.0-ompi415//lib:/central/software/OpenMPI/4.1.5//lib:/central/software/ucx/1.13.1_cuda-11.2//lib:/central/software/CUDA/11.2/lib64:/central/software/CUDA/11.2/extras/CUPTI/lib64:/central/software/CUDA/11.2/targets/x86_64-linux/lib		(LD_LIBRARY_PATH)
     21111:	  trying file=/central/software/julia/1.8.5/lib/libpcre2-8.so
     21111:	  trying file=/central/slurm/install/current/lib/libpcre2-8.so
     21111:	  trying file=/central/software/hdf5/1.14.0-ompi415//lib/libpcre2-8.so
     21111:	  trying file=/central/software/OpenMPI/4.1.5//lib/libpcre2-8.so
     21111:	  trying file=/central/software/ucx/1.13.1_cuda-11.2//lib/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/lib64/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/extras/CUPTI/lib64/libpcre2-8.so
     21111:	  trying file=/central/software/CUDA/11.2/targets/x86_64-linux/lib/libpcre2-8.so
     21111:	 search cache=/etc/ld.so.cache
     21111:	  trying file=/lib64/libpcre2-8.so
     21111:	
     21111:	
     21111:	calling init: /lib64/libpcre2-8.so

From what I can tell, it is not attempting to search the RUNPATH for libpcre2-8.so, so instead searches the cache and ends up with the wrong library.

This was with Nsight Systems 2023.2.1, on CentOS Linux.

2

As an addendum, it does appear that RUNPATH is used for other libraries (see full output here LD_DEBUG output.txt · GitHub), but i’m not sure why not for libpcre

3

I’ve found I can fix it if I exclude osrt from the --trace option.

4

I ran into the same issue, on my system I had to disable osrt and opengl.