Cannot show nvcc --version

Hi,
I am implementing about cross compile on x86 ubuntu for Jetson Xavier ubuntu18.04
According to official documentation about cross compile, I clone the filesystem from Xavier.
I mount the clone.img.raw by sudo mount -t ext4 clone.img.raw jetson and it works
but when I cd to /jetson/usr/local/cuda-10.2/bin and execute ./nvcc --version
it shows

~/jetson/usr/local/cuda-10.2/bin$ ./nvcc --version
./nvcc: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory

it will damage my cross compile project because I will use the cuda package in this file system
how do I solve this problem?

Hi,

The error indicates some missing library on your environment.
Do you also clone the lib for cross-compiling?

Thanks.

Incidentally, what shows up from this command:
ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

Note: If you are using a cross linker, then you need to run that particular ld, not the default one. This means you might be naming the full path to the cross linker. An example might be:
/usr/aarch64-linux-gnu/bin/ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
OR:
/usr/bin/aarch64-linux-gnu-ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

I actually clone the whole filesystem from Xavier so I do have aarch 64 lib in the jetson I mount by clone.img.raw.

I deduce that nvcc in jetson can’t find libpthread.so.0 from jetson mounted by clone.img.raw
I am not sure it’s right deduction.

Hi I use a cross linker gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu
and as you said I run the command you gave, the result like below

SEARCH_DIR("=/home/tcwg-buildslave/workspace/tcwg-make-release/builder_arch/amd64/label/tcwg-x86_64-build/target/aarch64-linux-gnu/_build/builds/destdir/x86_64-unknown-linux-gnu/aarch64-linux-gnu/lib64")
SEARCH_DIR("=/usr/local/lib64")
SEARCH_DIR("=/lib64")
SEARCH_DIR("=/usr/lib64")
SEARCH_DIR("=/home/tcwg-buildslave/workspace/tcwg-make-release/builder_arch/amd64/label/tcwg-x86_64-build/target/aarch64-linux-gnu/_build/builds/destdir/x86_64-unknown-linux-gnu/aarch64-linux-gnu/lib")
SEARCH_DIR("=/usr/local/lib")
SEARCH_DIR("=/lib")
SEARCH_DIR("=/usr/lib")

How do I add particular path into the SEARCH_DIR of ld

Before answering, I want to add that a cross linker needs native libraries for its own execution (the cross linker executes in an amd64/x86_64 environment), plus a linker path to the arm64/aarch64 libraries for which linking is actually provided (the target executable being linked is arm64/aarch64). This talks about the libraries being linked and not the libraries used for the execution environment.

The most common method is the “-L path” option, which is basically the same as any other linker (except configuration files for setting up the default will not be the same as the native linker). For example, if you have this directory with “~/clone/usr/lib/aarch64/*” library files:
-L ~/clone/usr/lib/aarch64

There is a “--sysroot=<DIRECTORY>” option which can be used to change the entire sysroot (and sysroot is basically what the cross tools call the start of the filesystem containing the foreign architecture files). This can differ with different toolchains, but should be correct for anything from Linaro or what NVIDIA uses.

I found libpthread.so.0 under /home/username/jetson/usr/lib/aarch64-linux-gnu
and add the path into LD_LIBRARY_PATH environment variable

export LD_LIBRARY_PATH=/home/username/jetson/lib/aarch64-linux-gnu:$LD_LIBRARY_PATH

and then go back to /home/username/jetson/usr/local/cuda-10.2/bin to command
./nvcc --version, it works

could anyone explain why?

Almost every Linux system (and non-Linux system, but this is for Linux) tries to be efficient and reliable by making available a unified set of libraries which any application can use for some standard function. One could compile that content into every executable, but it would be a waste of space if the content of shared libraries could be put on disk once, and each application copy that subset of its program into RAM as it starts. That’s what dynamic libraries are.

As such, a standardized mechanism has to be available for finding and loading that content. Think of the security issues and performance problems if every program implemented its own search for that content. The mechanism for finding and assisting in loading shared content is the linker. The linker itself has a search path, and if a specification for a library is presented to the linker, then it finds the best choice within that search path. If the library in question is not found, then it fails.

The ld_library_path is one specification of where to search. You can append to it each compile, or if the content is in a location which will be accessed by several programs, then that directory can be appended to the default search path to make it more convenient.

That’s all “runtime”. At the time you compile a program there are additional problems. The function calls to libraries have to follow a predefined application binary interface scheme (the ABI). This basically matches the called function signature to the library’s provided function signatures. The binary compiled program will work with library calls with a signature that matches the actual location and signature in memory address which the program expects. So during compile time the program sets aside an address span for the library to load at (with the help of the linker). This becomes etched into the binary executable.

In the case of a cross linker it adds yet another issue. The linker program itself needs libraries to operate which are native to the platform. However, the code of the function call is a different ABI than what is native (think of arm64/aarch64 assembler instead of amd64/x86_64 assembler). So now you need:

  • The linker’s execution environment library calls for it to function.
  • The signature search in arm64/aarch64 ABI.
  • A compiler creating an address span compliant with the same arm64/aarch64 ABI.

The linker ties all of that together. If you lack the native libraries (or lack the ability to find those libraries), then the linker fails. If you try to link a library which a compiler is looking for in order to design the load address, then the linker works, but tells you it didn’t find the content. If your linker works, and it finds the library being linked, but it is the wrong signature (in this case due to wrong architecture), then it once more fails to set up linking. If all of these are found, and ABIs are correct, then the binary executable creates that address span to load the signature of that library function.

When you set up your Jetson for development, and have all development content plus libraries on it, then your linker and cross linker have everything they need, but only if it can find them. You still have to tell the PC where the cross architecture libraries are found.

1 Like

Hi thanks for explanation. you’re very kind.
it makes me more clear on how the linker works !