Cross Compiling on AGX Xavier

[lukas.hann95]

Hi,

I am following the cross compilation instructions here, but I am having some issues. I am just trying to compile a sample for now, 00_video_decode

I have the Jetson set up, and I can compile the sample (00_video_decode) on the target.

I have successfully cloned the image to my host machine, and mounted it.

I have downloaded the toolchain (64 bit, version 28.3.2), but this is where my confusion begins. According to the instructions, I am supposed to:

export PATH=<CROSS_COMPILER_PATH>/bin:$PATH

but the file I downloaded contains both install/bin and install/aarch64-unknown-linux-gnu/bin. Which of these am I supposed to use?

I have tried both, but neither worked.

install/bin

When following the steps, I get the error

aarch64-linux-gnu-g++: not found

because the compiler in install/bin is called aarch64-unknown-linux-gnu-g++. I had a look at the makefile and realized I had to change the CROSS_COMPILE env from

export CROSS_COMPILE=aarch64-linux-gnu-

to

export CROSS_COMPILE=aarch64-unknown-linux-gnu-

Now I get the error

Compiling: video_decode_csvparser.cpp
aarch64-unknown-linux-gnu-g++: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
Aborted (core dumped)
make: *** [Makefile:49: video_decode_csvparser.o] Error 134

After some googling, I found a fix: export LANG=C. This gets rid of the error, but I do not understand why. If anybody could explain it, that would be great :)

Now I can compile the sample, but the linker fails

Compiling: video_decode_csvparser.cpp
Compiling: video_decode_main.cpp
make[1]: Entering directory '/home/lhahn/Documents/CrossCompilation/Images/jetson/usr/src/jetson_multimedia_api/samples/common/classes'
Compiling: NvElementProfiler.cpp
Compiling: NvElement.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvBuffer.cpp
Compiling: NvLogging.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2Element.cpp
make[1]: Leaving directory '/home/lhahn/Documents/CrossCompilation/Images/jetson/usr/src/jetson_multimedia_api/samples/common/classes'
Linking: video_decode
/home/lhahn/Documents/CrossCompilation/gcc-4.8.5-aarch64/install/bin/../lib/gcc/aarch64-unknown-linux-gnu/4.8.5/../../../../aarch64-unknown-linux-gnu/bin/ld: cannot find crt1.o: No such file or directory
/home/lhahn/Documents/CrossCompilation/gcc-4.8.5-aarch64/install/bin/../lib/gcc/aarch64-unknown-linux-gnu/4.8.5/../../../../aarch64-unknown-linux-gnu/bin/ld: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [Makefile:53: video_decode] Error 1

I have found the missing file in install/aarch64-unknown-linux-gnu/sysroot/usr/lib/crti.o and in the cloned image ${ROOT_FS}/usr/lib/aarch64-linux-gnu/crti.o. I checked the g++ search paths (aarch64-unknown-linux-gnu-g++ -print-search-dirs) which are passed to the linker, and they do not include either of the 2 locations above. I don’t understand where I went wrong, and manually hacking it in here seems wrong. Can someone please tell me how to fix this properly?

install/aarch64-unknown-linux-gnu/bin

When following the steps, I get the same initial error as before,

/bin/sh: 1: aarch64-linux-gnu-g++: not found

because the compiler in install/aarch64-unknown-linux-gnu/bin is simply called g++, so I cleared the CROSS_COMPILE env. Now the Rules.mk automatically set the CROSS_COMPILE env to aarch64-unknown-linux-gnu-. I uncommented that part.

Now I get the same error as above,

Compiling: video_decode_csvparser.cpp
g++: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
Aborted (core dumped)
make: *** [Makefile:49: video_decode_csvparser.o] Error 134

That’s fixed by

export LANG=C

Now I get the error

Compiling: video_decode_csvparser.cpp
g++: error trying to exec 'cc1plus': execvp: No such file or directory
make: *** [Makefile:49: video_decode_csvparser.o] Error 1

I tried googling this as well, but didn’t find anything relevant.

I would really appreciate some help with this.

Kind regards,

Lukas

[linuxdev]

I have not gone through all of those docs, but thought I’d just mention some basics which might make sense…

Whenever you use a cross compiler, then typically the “gcc” app is used, but it is renamed based on architecture. An example is that on a PC you might compile natively with gcc, but cross compile to 64-bit ARM via “aarch64-linux-gnu-gcc”. The difference is that gcc runs on the host PC, and also produces native output for the host PC; the aarch64-linux-gnu version runs on the PC, but produces cross architecture output.

The name “aarch64-linux-gnu-” will usually prefix all of the programs associated with gcc intended for this cross architecture output. So you might have a directory like this:
/usr/aarch64-linux-gnu
…and within this (this isn’t a hard rule, it is just typical), a directory for each release of gcc cross tools (many people will just use the release which comes with Ubuntu, I have many releases, and so I have many such directories). Each release will in turn have a bin/ directory. In most cases this is where you would find “aarch64-linux-gnu-gcc”. You would also find a cross linker (links aarch64 libraries, but does so from a PC) there, e.g., “aarch64-linux-gnu-ld”. When you specify a CROSS_COMPILE path, what this does is specify a prefix path to the aarch64 version.

As an example, if you specify “CROSS_COMPILE=/usr/bin/aarch64-linux-gnu-”, then when compile wants “gcc”, it will prefix this, and the full path becomes “/usr/bin/aarch64-linux-gnu-gcc”. Similarly, with that CROSS_COMPILE, a request for ld (the linker) would look for "/usr/bin/aarch64-linux-gnu-ld". *You should think of CROSS_COMPILE as a prefix to prepend to what would otherwise be a basic gcc tool*. If for some reason your cross tools don't have "aarch64-linux-gnu-" as a prefix, then your CROSS_COMPILE would still need thebin/ directory of the tools, but it wouldn't have the "aarch64-linux-gnu-`" prefix.

When you build “bare metal” applications (e.g., kernels or bootloaders…these don’t link to libraries) there is no need for a linker. Once you go into “user space” you will probably need to link software with ld. Even if you have the cross tools for linking, this does not provide the actual libraries to link against. Somewhere you’d have an aarch64-linux-gnu subdirectory for lib/ files. You’d have to correctly name the location to find those aarch64 files (the linker would use that information).

Libraries are typically not shipped with tools. These are usually a separate install called the “sysroot”. So for example you might have a “/usr/lib/aarch64-linux-gnu/” directory, and none of those files would work on the host PC; only the cross ld could use those since it can run on the PC, but expects aarch64 libraries.

The sysroot is often available only with the most basic of requirements, and thus often not suitable by itself if you plan to link against less common libraries. For this I tend to recommend to set up the Jetson (if cross compiling from the PC for the Jetson) to be able to compile with all of the correct development tools and environment, and then to clone this, followed by mounting the clone on the host PC. The actual libraries (the sysroot) would then be used by naming the library location on the clone.

If you can build a Linux kernel via cross compile, then you know your basic tool chain is set up correctly. I recommend this as a test. Once that can be done, and only once that works, try setting up and building a user space application. This latter uses everything of the former, but introduces linking.

[DaneLLL]

Hi,
For r28 releases, please refer to:
L4T Multimedia API Reference: Setting Up Cross-Platform Support

[lukas.hann95]

Thank you for noticing my mistake, I did not notice the version mismatch. I just downloaded the most recent version. It is a bit weird that the 28 release came after the 32 release. Anyways, I fixed it by getting the version 32 toolchain rather than using the older instructions for the 28 release. Thank you for your help!