CMake Cross-Compiling Linker Errors

Robotics & Edge Computing Jetson Systems Jetson TX2
1

Hey,

Leading with this:
I’ve been able to successfully build the application on the TX2i but because we’re contractually obligated to provide cross-compiling support, I need to be able to cross-compile.

I’ve been trying to cross-compile an application for the TX2i using CMake and it always fails to link.

I’ve downloaded the latest cross-compile toolchain (GCC Tool Chain for 64-bit BSP) and unzipped it into my x86_64 machine’s /home/user/ directory.

Then I followed LinuxDev’s guide here to create a loopback copy of my TX2i files on the x86_64 machine.

Then I created these symbolic links:

/usr/lib/aarch64-linux-gnu → /usr/local/loopback_tx2i/usr/lib/aarch64-linux-gnu
/usr/include/aarch64-linux-gnu → /usr/local/loopback_tx2i/usr/include/aarch64-linux-gnu

I managed to cross-compile a simple hello_world program with this command:

cmake -D CMAKE_TOOLCHAIN_FILE=…/toolchain.cmake …

Here is the toolchain.cmake file:

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)
set(CMAKE_LIBRARY_ARCHITECTURE aarch64-unknown-linux-gnu)

set(CMAKE_SYSROOT /usr/local/loopback_tx2i)

set(CMAKE_C_COMPILER /home/user/install/bin/aarch64-unknown-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER /home/user/install/bin/aarch64-unknown-linux-gnu-g++)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)

set(CMAKE_TRY_COMPILE_TARGET_TYPE “STATIC_LIBRARY”)

I copied the executable over to my TX2i and it printed Hello World! as expected.

However, when I try to cross-compile my actual application with the same toolchain file, I get the following output.

Scanning dependencies of target [redacted]
[ 10%] Building CXX object src/CMakeFiles/[redacted].dir/[redacted].cpp.o
[ 20%] Building CXX object src/CMakeFiles/[redacted].dir/home/user/[redacted]/application/tcp_msgs/cpp_msgs/[redacted].pb.cc.o
[ 30%] Building CXX object src/CMakeFiles/[redacted].dir/home/user/[redacted]/application/tcp_msgs/cpp_msgs/[redacted].pb.cc.o
[ 40%] Building CXX object src/CMakeFiles/[redacted].dir/[redacted].cpp.o
[ 50%] Building CXX object src/CMakeFiles/[redacted].dir/I2CInterface.cpp.o
[ 60%] Linking CXX executable [redacted]
/home/user/install/bin/…/lib/gcc/aarch64-unknown-linux-gnu/4.8.5/…/…/…/…/aarch64-unknown-linux-gnu/bin/ld: warning: libz.so.1, needed by /usr/local/loopback_tx2i/usr/local/lib/libprotobuf.so, not found (try using -rpath or -rpath-link)
CMakeFiles/[redacted].dir/[redacted].cpp.o: In function main': [redacted].cpp:(.text+0x4cc): undefined reference to google::protobuf::MessageLite::SerializeAsString() const’
CMakeFiles/[redacted].dir/[redacted].cpp.o: In function `google::protobuf::internal::GetEmptyStringAlreadyInited()':

[redacted].cpp:(.text._ZN6google8protobuf8internal27GetEmptyStringAlreadyInitedEv[_ZN6google8protobuf8internal27GetEmptyStringAlreadyInitedEv]+0x8):
undefined reference to ` google::protobuf::internal::fixed_address_empty_string’
…Nearly 100 more lines of linker errors…

I’m assuming my CMake toolchain file is just not pointing to the right libraries or I haven’t created enough symbolic links.

Any help is appreciated.

Thank you,
David

2

Everything is actually working, but you apparently did not have libz.so.1 installed on the loopback image (or perhaps the search path was not correct to find libz.so.1).

If you are on your running Jetson which the clone came from, here are some things to examine:

  • dpkg -l | grep -i zlib
  • apt search zlib
  • ldconfig -p | grep -i 'libz[.]'

Notice that the base zlib packages will provide libraries, and that devel versions are not required for linking. However, if you are building an application which links against this, then there is a possibility you will also need to devel package to provide headers in addition to the actual libraries.

If you were to install this on the Jetson, you could then use a number of methods to update the clone instead of creating a new clone, e.g., rsync or just manual file copies. In your situation it might be worth your time to work through the learning curve of rsync, but if you are in a hurry, the you could also simply clone again. Either way be sure to save an unmodified version of your clone in case experiments with manual file copy or rsync automated update don’t go well.

An acid test to know if the clone image is right would be if you can compile that software directly on the Jetson itself. If you can do this, then your use of a clone in cross compile is guaranteed to have all of the required user space libraries and files, and then all else is just cross compile setup.

Additional notes: Usually search paths, if relative, implies that a full clone with loopback mount, where that mount point is the top level directory, will “just work”. The clone is a standard layout other than being mounted at some offset mount point (it is no longer on “/”, and only that needs to be updated if paths are specified as a base/top directory, plus standard relative paths relative to that top level directory).

3

Thank you, its good to know I properly mounted the image.

I’m not entirely sure what I’m suppose to be looking for from those commands but it seems to be installed. I am able to compile and run the applicaiton on the TX2i so there must be something wrong in my cross-compile setup.

I grepped for libz.so.1 in my mounted image and found it here:
<loopback_mount>/lib/aarch64-linux-gnu/libz.so.1

Creating a symlink to it in <loopback_mount>/lib/libz.so.1 fixed that one issue but there are still protobuf linking errors which makes me think the cross-compiler isn’t looking in the right place and the symlink is just a band-aid.

I’m going to try adding that lib path to my cmake_library_path but if you have any more ideas then please let me know.

4

There are a lot of tools and other software which assumes you are running on the same system that you are compiling from. It isn’t surprising if you need to include a lot of sym links. In cases where tools are cross-compile away there may be options to offset all paths by the base path of the image being mounted, but if you were to compile natively on the Jetson (which is a good acid test…does this same thing build correctly natively on a Jetson?), then there is still a significant chance that compile might not find everything it needs without a lot of intervention to add the correct path.