Cross-compile linking libgomp library failed

[120090862]

Env

ubuntu 22.04
toolchain use: Jetson Linux Toolchain — Jetson Linux<br/>Developer Guide 34.1 documentation
toolchain locates in /usr/local/toolchain with correct $PATH in ~/.bashrc
also

export CROSS_COMPILE=/usr/local/toolchain/bin/aarch64-buildroot-linux-gnu-

Jetson Env

Ubuntu 20.04
GCC 9.4
Glibc 2.31

Background

I am using AGX Orin to deploy a .so library developed from x86 architecture, so I am using the L4T toolchain provided by NVIDIA for cross-compiling (host: x86 target: Jetson Orin). However, an error occurs when I try to use the libgomp library: even though the ‘libgomp.spec’ file and relevant files exist in the toolchain directory, I still get the error message:
aarch64-buildroot-linux-gnu-g++.br_real: fatal error: cannot read spec file 'libgomp.spec': No such file or directory

‘omp.h’ file cannot be recognized by makefile at first but can be solved by adding -I /path/to/omp.h/inside/toolcahin

Using toolchain provided by arm do solve this problem but it provide me with different glibc version.

I am not familiar with c++ and makefile, so I will post the whole makefile I use below incase I miss something. Any suggestion or help will be appriciated!

Error log

aarch64-buildroot-linux-gnu-g++ -I./ubnet -I./ubnet/3rd_party -I./ubnet/layers -I./public_interface/ -I/usr/local/toolchain/aarch64--glibc--stable-final/lib/gcc/aarch64-buildroot-linux-gnu/9.3.0/include/ -Wall -std=c++11 -pthread -fPIC -shared -march=armv8-a+simd -L /usr/local/toolchain/aarch64--glibc--stable-final/aarch64-buildroot-linux-gnu/lib64/ -g -O0 -fopenmp  -o utts.so tts_offline_lib.o public_interface/func.o questions_set/questions.o ubnet/helpers/weight_io.o ubnet/functions.o ubnet/layers/blstm.o ubnet/layers/layer.o ubnet/layers/linear.o duration/duration_predict.o duration/dur_model.o duration/dur_net.o acoustic/aco_model.o acoustic/aco_net.o acoustic/acoustic_predict.o htslabel/label_encoder.o 
aarch64-buildroot-linux-gnu-g++.br_real: fatal error: cannot read spec file ‘libgomp.spec’: No such file or directory

What I have tried

1. add ‘lgomp’

#compile command
$(BIN) : $(OBJS)
        $(CXX) $(CXX_FLAGS) -o $@ $^ -lgomp

2. use -I and -L to link libgomp inside toolchain

#compile command
$(BIN) : $(OBJS)
        $(CXX) $(CXX_FLAGS) -o $@ $^ -L /usr/local/toolchain/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/lib64/ -lgomp

Makefile I use

#platform
#CXX = aarch64-none-linux-gnu-g++
#CXX = aarch64-linux-g++
CXX = aarch64-buildroot-linux-gnu-g++
#INCS = -mavx -mfma
RM = rm -f

export IS_MAKE_SO = TRUE

#define .h
INCS = -I./ubnet -I./ubnet/3rd_party -I./ubnet/layers -I./public_interface/ -I/usr/local/toolchain/aarch64--glibc--stable-final/lib/gcc/aarch64-buildroot-linux-gnu/9.3.0/include/

#define *.cpp
CPPSRCS = $(wildcard ./*.cpp ./public_interface/*.cpp)
CPPSRCS += $(wildcard ./questions_set/*.cpp ./ubnet/helpers/*.cpp ./ubnet/*.cpp ./ubnet/layers/*.cpp  ./duration/*.cpp ./acoustic/*.cpp ./htslabel/*.cpp)

#define objs
CPPOBJS := $(CPPSRCS:.cpp=.o)
OBJS := $(CPPOBJS)

#define dependence
CPPDEF := $(CPPSRCS:.cpp=.d)


CXX_MAKEDEPEND := $(CXX) -MM -std=c++11

#define g++ param
ifeq ($(IS_MAKE_SO), TRUE)
        CXX_FLAGS = $(INCS)
        CXX_FLAGS += -Wall -std=c++11 -pthread -fPIC -shared
        CXX_FLAGS += -march=armv8-a+simd -L /usr/local/toolchain/aarch64--glibc--stable-final/aarch64-buildroot-linux-gnu/lib64/
        # define final target
        BIN := ./utts.so
else
        CXX_FLAGS = $(INCS)
        CXX_FLAGS += -Wall -std=c++11 -pthread
        # define final target
        BIN := ./dist/tts_server
endif


CXX_MAKEDEPEND := $(CXX) -MM -std=c++11

debug : CXX_FLAGS += -g -O0 -fopenmp #-lprofiler -ltcmalloc  -lprofiler, -ltcmalloc = gperftools
debug : all

release : CXX_FLAGS += -Ofast -DNDEBUG -fopenmp #-lprofiler -ltcmalloc #-DEIGEN_DONT_PARALLELIZE
release : all

all : $(BIN)


$(CPPDEF) : %.d : %.cpp
        $(CXX_MAKEDEPEND) $< $(INCS) > $@

#compile command
$(BIN) : $(OBJS)
        $(CXX) $(CXX_FLAGS) -o $@ $^ 

$(CPPOBJS) : %.o : %.cpp
        $(CXX) $(CXX_FLAGS) -c $< -o $@


-include $(CPPDEF)

#clean command
.PHONY : clean cleanall debug release all
clean :
        @$(RM) $(OBJS) $(CPPDEF)

cleanall :
        @$(RM) $(BIN) $(OBJS) $(CPPDEF)

[linuxdev]

I cannot help with details. I do have a suggestion though. What you’re needing is part of the “sysroot”. When you build bare metal, e.g., a kernel or a bootloader, there is nothing to link against, and so no sysroot is needed. You’re in user space though, so you need the entire set of every library or linker the compiler might look for during link stage. One of the best ways to do this is to set up an actual Jetson with the ability to build this same software, and then clone the rootfs partition. The raw version of the clone can be mounted on loopback on the host PC, and named as the sysroot. This makes everything that the user space Jetson could build available on the host PC.

Before the actual clone you’d want to first update packages, and then add any dev version of libraries which might be required. The actual clone could be an rsync, but a true clone also serves as a backup. There would be both a “raw” clone and a “sparse” clone; I throw away the sparse clone. The raw clone is a huge file, the size of the entire partition, so you’d need enough space (during clone both a raw and a sparse partition clone will be produced…the sparse size is the size of the actual content, and as the partition fills up, the sparse size approaches the size of the partition).

You could also find out what provides that file, install it on the Jetson, and then rsync just that content.