[CrossCompile] Error -march parameter

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hello everyone,

I have been struggling with cross compiling for my TX2 for a few weeks and there is an issue I have been unable so solve (been only able to bypass it with nasty scripts).

Setup:
An host with ubuntu 16.04 and the Jetpack 3.2.1 installed. Also installed the cross compiler aarch64.
A TX2 board flashed and set-up according to my host.

Compilers:
Board: gcc --version = 5.4.0
Host : aarch64-linux-gnu-gcc -v = 5.4.0 (gcc --version = 5.4.0)

I am performing the build & install with CMake so I set up a large quantity of variables in order to position everything properly.

But I am getting stuck with this error :

error: unknown value ‘native’ for -march

After deep investigation (lots of generated makefile reading) it turns out that the march flag actually doesn’t work with the current cross-compiler (aarch64-linux-gnu) so the error make sense.

What doesn’t; is that even though I try to force these compiler flags into all the CMake files, the option -march=native stays first and trigger the error.

My toochain file starts with:

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 16.04)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(RECOMMENDED_TX2_FLAGS "-O3 -ffast-math -flto -march=armv8-a+crypto -mcpu=cortex-a57+crypto")
...
set(CMAKE_CXX_FLAGS "-I${TARGET_FS}/usr/include -I${TARGET_FS}/usr/include/aarch64-linux-gnu")
...

The only way I found to bypass this error is actually pretty simple but it doesn’t “feel” right:

function correctCmakeFlags() {
    flagFile=$(find ./ -type f -name "flags.make")
    for f in $flagFile; do
        sed -i.bak 's/-march=native/-march=armv8-a/g' $f
    done
}

I guess as some people are cross-compiling without any issue, there is probably some parameter that either I missed or misinterpreted, is there anyone having the same issue and solved it ?
Thanks !

2

If you compile single threaded (“-j1”) to avoid seeing several operations being mixed together, and compile as much as possible before running with “-j1” again, what do you see logged? You can use the “code” icon (looks like “</>” when editing your post) and paste directly into that, or else attach it as a file with a “.txt” suffix.

3

With a make -j1 option the error is exactly the same, on the first *.cpp file (in terms of dependency order) the error occur.

Input:

declare -r TARGET_PROCESSOR=arm-v8
declare -r TOOLCHAIN_FILE=toolchain.cmake
cd build/
cmake .. -D"CMAKE_TOOLCHAIN_FILE"=${TOOLCHAIN_FILE} \
    -D"TARGET_PROCESSOR"=${TARGET_PROCESSOR} \
    -D"CMAKE_EXPORT_COMPILE_COMMANDS"=ON \
    -D"CMAKE_BUILD_TYPE"=Debug \
    ${SOURCE_LOCATION}
make -j1

Output:

Building NVCC (Device) object src/Kernel/processing.cu.o
Scanning dependencies of target initial 
[25%] Building CXX object src/initial_values.cpp.o
/home/user/code/src/initial_values.cpp:1:0: error: unknown value 'native' for -march
/**
^
build.make:69: recipe for target 'src/initial_values.cpp.o' failed

What is strange with the -j1 argument is that it seems to reach some point in building before throwing the error…
It seems that when it starts reaching CUDA code, the build fails…

I will investigate more deeply my CMake script building CUDA, I might have an issue here, I actually don’t really know how I should position my flags for CUDA TX2 cross-compiling.

4

Unfortunately your cmake isn’t echoing the actual build command. You might be interested in this to get echo enabled:
[url]linux - Making CMake print commands before executing - Stack Overflow

The more verbose the better. With enough verbosity you might be able to see a variable name which is containing the “-march native”.

5

Hello, thanks for to advise. Actually, that was something I already did in order to program my nasty script up there. Instead of having an “overly-verbose” console (which sometime make things hard to find) I used the

-D"CMAKE_EXPORT_COMPILE_COMMANDS"=ON

Command which allow me to have all the compile commands exported into “compile_commands.json” file.

Anyway, I did what you recommended and found the full command giving the error:

cd src/Kernel && /usr/bin/aarch64-linux-gnu-g++  --sysroot=/mnt/nvidia_tx2  -DDISABLE_LIBUSB_1_0 -DDISABLE_PCAP -DDISABLE_PNG -Dmain_EXPORTS -Dqh_QHpointer -isystem /mnt/nvidia_tx2/usr/local/include/pcl-1.8 -isystem /mnt/nvidia_tx2/usr/include/eigen3 -isystem /mnt/nvidia_tx2/usr/include/opencv -I/mnt/nvidia_tx2/usr/local/cuda-9.0/include  -D_FORCE_INLINES -O3 -DNDEBUG -fPIC  -march=native -ffloat-store -std=gnu++11 -o CMakeFiles/Kernel.dir/src/initial_values.cpp.o -c /home/localuser/code/src/Kernel/initial_values.cpp

As you can see I’m depending on different external libraries. In order to include them, I mounted the board as a folder in “/mnt” and included them.

I was expecting this to work (and it does with nasty script) but I don’t fully understand why is the “native” value isn’t working.

I checked other references to get a better view of the problem and from what I understood, changing the compiler flags should do the trick. But when I go into my toolchain file and change the flags, they just add-up to the left side of the Makefile, keeping the “-march=native” triggering the error.

[...]-D_FORCE_INLINES -march=armv8-a -O3 -DNDEBUG -fPIC   -march=native [...]

In the end, the information I am looking for is “Where can I override the CMake compiler flags while cross-compiling when I’m using CUDA among the process ?”

6

You’re cross compiling, and “/usr/bin/aarch64-linux-gnu-g++” supports only one architecture, so probably you can skip “-march=…”. If not, then “native” is not an architecture when cross compiling. “aarch64” or “arm64” would be valid if you need to state “-march”. I’m unsure of the CMake cross compile overrides, someone else can probably help with that. If it were a kernel compile you just “export ARCH=arm64” prior to starting the build, or else a literal “ARCH=arm64 aarch64-linux-gnu=g++ …” preceding the build line. Sorry, I don’t use CMake much.

7

Very nice ! Thanks a lot for the advice, this allowed me to solve the issue. I didn’t knew that this flag could be totally removed without any side-effect…

In fact the script responsible of setting this parameter was out of my scope (actually PCLConfig.cmake which was locally compiled -and not cross-compiled-).

When locally compiled, it does set the flags:

list(APPEND PCL_COMPILE_OPTIONS -march=native -ffload-store)

And later, when cross compiling, the remote cmake set your flag, triggering the error. The issue is specific to our particular configuration we got here.

Good solution: cross-compile your libraries from sources
Bad solution: simulate the cross compiling flag option (even though you have not).