Сrosscompilation error on JTX1

maxfashko · September 5, 2017, 8:52am

Hey. I experienced a lot of pain and suffering. I had difficulties with cross-compiling to my JTX1 R28.1 (Qt 5.9). I followed this instruction https://www.innovative-dsp.com/ftp/jetson/Configure%20Cross%20Compiling%20for%20Nvidia%20Jetson%20TX2%20on%20Ubuntu%2064%20bit.pdf.

The original script https://www.innovative-dsp.com/ftp/jetson/Host_Setup.sh did not work and I fixed it:

#!/bin/bash
#check for proper usage
if [ "$#" -ne 1 ]; then
    echo "usage ./Host_Setup.sh IP_ADDRESS_OF_JETSON_DEVICE"
	echo "example:"
	echo "./Host_Setup.sh 192.168.0.101"
    exit -1
fi

IP_ADDRESS=$1

#Install the GCC ARM toolset Linaro **VERSION 5
sudo apt-get install -y gcc-5-arm-linux-gnueabi g++-5-arm-linux-gnueabi

#create 64TX2 dir
cd $HOME && mkdir 64_TX2
mkdir 64_TX2/Linux_for_Tegra_tx2
mkdir 64_TX2/Linux_for_Tegra_tx2/rootfs

#Download GCC Linaro source
cd $HOME/64_TX2
sudo wget --no-check-certificate http://releases.linaro.org/components/toolchain/binaries/5.4-2017.05/aarch64-linux-gnu/gcc-linaro-5.4.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz
sudo tar -xvf gcc-linaro-5.4.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz

#Download Qt Source (Get at least Qt version 5.9.0)
cd $HOME
sudo wget --no-check-certificate http://download.qt.io/archive/qt/5.9/5.9.0/single/qt-everywhere-opensource-src-5.9.0.tar.xz
sudo tar -xvf qt-everywhere-opensource-src-5.9.0.tar.xz

#Install prerequisite Ubuntu packages for compiling Qt
sudo apt-get install -y 'libxcb.*' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev 'libinput*' 'mtdev*'  mesa-utils mesa-utils-extra libgles2-mesa-dev

#Sync files from device back to host ROOTFS
cd $HOME/64_TX2/Linux_for_Tegra_tx2/rootfs
sudo rsync -e ssh -avz ubuntu@$IP_ADDRESS:/usr/include ./usr
sudo rsync -e ssh -avz ubuntu@$IP_ADDRESS:/usr/lib ./usr

#Fix Symbolic Links
cd $HOME
sudo wget --no-check-certificate https://www.innovative-dsp.com/ftp/jetson/fixsymlinks.sh
sudo chmod +x ./fixsymlinks.sh
sudo ./fixsymlinks.sh $HOME/64_TX2/Linux_for_Tegra_tx2/rootfs

#Configure Qt with these options
cd $HOME/qt-everywhere-opensource-src-5.9.0
./configure -opensource -release --confirm-license -device linux-jetson-tx1-g++ -device-option CROSS_COMPILE=$HOME/64_TX2/gcc-linaro-5.4.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- -sysroot $HOME/64_TX2/Linux_for_Tegra_tx2/rootfs -nomake examples -nomake tests -prefix /usr/local/qt5 -extprefix $HOME/64_TX2/qt5 -hostprefix $HOME/64_TX2/qt5-host -opengl es2 -L /usr/lib/aarch64-linux-gnu -skip webkit -skip webview -skip webkit-examples


#Make Qt (the -j8 switch uses 8 CPU cores to do the build instead of the default 1 core)
make -j8

#Install Qt
sudo make install #Note: this will install the Qt build to /home/<USERNAME>/64_TX2/qt5-host

#Sync newly built Qt with Jetson device
cd $HOME/64_TX2/
sudo rsync -e ssh -avz qt5 ubuntu@$IP_ADDRESS:/usr/local

Line

#Install prerequisite Ubuntu packages for compiling Qt
sudo apt-get install -y 'libxcb.*' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev 'libinput*' 'mtdev*'  mesa-utils mesa-utils-extra libgles2-mesa-dev

prints an error:

Packages with unsatisfied dependencies:
libinput10-dbg: Depends: libinput10 (= 1.2.3-1ubuntu1) but 1.6.3-1ubuntu1 ~ 16.04.1 will be installed
E: It is impossible to fix errors, you have held broken packages.

I did not correct this error and continued the build phase. In the script https://www.innovative-dsp.com/ftp/jetson/Device_Setup.sh i change user:

#!/bin/bash
#Install Ubuntu packages
sudo apt-get install '.*libxcb.*' libxrender-dev libxi-dev libfontconfig1-dev libudev-dev

#Get GLES and EGL headers
mkdir /home/ubuntu/GLES
cd /home/ubuntu/GLES
apt-get download libgles2-mesa-dev
ar x libgles2*.deb
tar -xvf data.tar.xz
mkdir /home/ubuntu/EGL
cd /home/ubuntu/EGL
apt-get download libegl1-mesa-dev
ar x libegl1*.deb
tar -xvf data.tar.xz

#Copy headers to proper locations
cd /home/ubuntu/GLES/usr/include
sudo cp -r GLES2 GLES3 /usr/include
cd /home/ubuntu/EGL/usr/include
sudo cp -r EGL KHR /usr/include

#Setup symbolic link for libGLESv2
sudo ln -s /usr/lib/aarch64-linux-gnu/tegra-egl/libGLESv2.so.2 /usr/lib/aarch64-linux-gnu/libGLESv2.so

#Setup folder to copy built Qt to later
sudo mkdir /usr/local/qt5
sudo chown ubuntu /usr/local/qt5

#Show IP Address
echo Please note your IP address for rsync on the host
hostname -I

After entering

sudo rsync -e ssh -avz ubuntu@$IP_ADDRESS:/usr/lib ./usr

, an error occurs:

rsync: send_files failed to open “/usr/lib/cups/backend/serial”: Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [generator=3.1.1]

I did not correct this error and continued the build phase. After entering

./configure -opensource -release --confirm-license -device linux-jetson-tx1-g++ -device-option CROSS_COMPILE=$HOME/64_TX2/gcc-linaro-5.4.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- -sysroot $HOME/64_TX2/Linux_for_Tegra_tx2/rootfs -nomake examples -nomake tests -prefix /usr/local/qt5 -extprefix $HOME/64_TX2/qt5 -hostprefix $HOME/64_TX2/qt5-host -opengl es2 -L /usr/lib/aarch64-linux-gnu -skip webkit -skip webview -skip webkit-examples

, an error occurs:

/config.log

> /home/maksim/64_TX2/gcc-linaro-5.3.1-2016.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ -Wl,-rpath-link,/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib -Wl,-rpath-link,/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu -Wl,-rpath-link,/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu/tegra -Wl,-rpath-link,/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/lib/aarch64-linux-gnu --sysroot=/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs -Wl,-O1 -fuse-ld=gold -o verifyspec verifyspec.o   -L/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib -L/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/lib/aarch64-linux-gnu -L/home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu -L/usr/lib/aarch64-linux-gnu
> /usr/bin/aarch64-linux-gnu-ld.gold: error: can not open /home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu/libm.so: no such file or directory
> /usr/bin/aarch64-linux-gnu-ld.gold: error: can not open /home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/lib/aarch64-linux-gnu/libc.so.6: no such file or directory
> /usr/bin/aarch64-linux-gnu-ld.gold: error: can not open /home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1: no such file or directory
> /home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu/crt1.o(.text+0x24): error: undefined reference to '__libc_start_main'
> /home/maksim/64_TX2/Linux_for_Tegra_tx2/rootfs/usr/lib/aarch64-linux-gnu/crt1.o(.text+0x28): error: undefined reference to 'abort'
> collect2: error: ld returned 1 exit status
> Makefile: 67: failed to execute the recipe for the purpose of "verifyspec"
> make: *** [verifyspec] Error 1

I copied the directory /lib from the device to the host to directory $HOME/64_TX2/Linux_for_Tegra_tx2/rootfs/lib . Ok, continued the build phase! After entering

make -j8

, an error occurs:

.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: readFromSocket (): error: undefined reference to ‘QRingBuffer :: reserve (long long)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: emitReadyRead (int): error: undefined reference to ‘QIODevice :: channelReadyRead (int)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: emitReadyRead (int): error: undefined reference to ‘QIODevice :: channelReadyRead (int)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: emitBytesWritten (long long, int): error: undefined reference to ‘QIODevice :: channelBytesWritten (int, long long)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: emitBytesWritten (long long, int): error: undefined reference to ‘QIODevice :: channelBytesWritten (int, long long)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocket :: writeData (char const *, long long): error: undefined reference to ‘QRingBuffer :: append (char const *, long long)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocket :: writeData (char const *, long long): error: undefined reference to ‘QRingBuffer :: append (char const *, long long)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: fetchConnectionParameters (): error: undefined reference to ‘QIODevicePrivate :: setReadChannelCount (int)’
.obj / qabstractsocket.o: qabstractsocket.cpp: function QAbstractSocketPrivate :: writeToSocket (): error: undefined reference to ‘QRingBuffer :: free (long long)’
.obj / qsocks5socketengine.o: qsocks5socketengine.cpp: function QSocks5SocketEnginePrivate :: _ q_emitPendingReadNotification (): error: undefined reference to ‘QRingBuffer :: clear ()’
.obj / qsocks5socketengine.o: qsocks5socketengine.cpp: function QSocks5SocketEnginePrivate :: _ q_controlSocketError (QAbstractSocket :: SocketError): error: undefined reference to ‘QRingBuffer :: clear ()’
.obj / qsocks5socketengine.o: qsocks5socketengine.cpp: function QSocks5SocketEngine :: read (char *, long long): error: undefined reference to ‘QRingBuffer :: read (char *, long long)’
.obj / qnativesocketengine_unix.o: qnativesocketengine_unix.cpp: function QNativeSocketEnginePrivate :: nativeSelect (int, bool, bool, bool, bool *) const: error: undefined reference to ‘qt_safe_poll (pollfd *, unsigned long, timespec const *)’
.obj / qlocalsocket_unix.o: qlocalsocket_unix.cpp: function QLocalSocket :: waitForConnected (int): error: undefined reference to ‘qt_safe_poll (pollfd *, unsigned long, timespec const *)’
.obj / qlocalserver_unix.o: qlocalserver_unix.cpp: function QLocalServerPrivate :: waitForNewConnection (int, bool *): error: undefined reference to ‘qt_safe_poll (pollfd *, unsigned long, timespec const *)’
.obj / moc_qdnslookup_p.o: moc_qdnslookup_p.cpp: typeinfo for QDnsLookupRunnable: error: undefined reference to ‘typeinfo for QRunnable’
collect2: error: ld returned 1 exit status
Makefile: 510: failed to execute the recipe for the target “…/…/lib/libQt5Network.so.5.9.0”
make [3]: *** […/…/lib/libQt5Network.so.5.9.0] Error 1
make [3]: exit the directory “/home/maksim/qt-everywhere-opensource-src-5.9.0/qtbase/src/network”
Makefile: 273: failed to execute the recipe for the “sub-network-make_first” target
make [2]: *** [sub-network-make_first] Error 2
make [2]: exit the “/home/maksim/qt-everywhere-opensource-src-5.9.0/qtbase/src” directory
Makefile: 49: failed to execute the recipe for the purpose of “sub-src-make_first”
make [1]: *** [sub-src-make_first] Error 2
make [1]: exit the “/home/maksim/qt-everywhere-opensource-src-5.9.0/qtbase” directory
Makefile: 82: failed to execute the recipe for the “module-qtbase-make_first” target

linuxdev · September 5, 2017, 1:25pm

I have a Fedora host, so I can’t use JetPack, but I see the PDF instructions you gave (go to [url]https://www.innovative-dsp.com/ftp/jetson/[/url], then look for cross compile PDF), use JetPack 3.0. Currently (and with R28.1) JetPack is R3.1. What makes this important is that some of the system setup and boot setup changed when migrating from the 3.x kernel series to the 4.x kernel series. Perhaps some aspect of the instructions are no longer an exact match (like I said though, I can’t use JetPack so I don’t know).

Your errors seem to be missing files to link against. If you are cross-compiling just a kernel, then you basically do not need anything in user space, e.g., you’re not linking against anything, it’s bare metal. Once you cross-compile user space applications you will need your cross-compile environment to support aarch64 linking. On top of this you will need anything your build (looks like Qt) requires linking against. Some people will download this via a package manager, or else install packages for the linker part of this foreign architecture support via Linaro’s “sysroot” tarball (at least this is what Linaro calls the user space linker support). You will also need to install any missing dependency/library you see in your compile…if you already have that dependency, then it simply means your build system needs to be told what the path is for finding those missing files during the link stage. I have not tried to cross-compile Qt so I don’t know any specifics.

One alternative to using a package manager to install foreign architecture dependencies is to install those dependencies on your Jetson (you’ll need them anyway), and then clone the Jetson…you can then loopback mount the clone and use symbolic links (or just a clever loopback mounting scheme) and make the linker use the clone itself for required linking support.

josejich · September 5, 2017, 4:20pm

Hi maxfashko,

I see that you said to have troubles integrating you package for the TX1, however in your script I see lots of references to the path

cd $HOME/64_TX2/

Are you on TX1 or TX2? Another alternative can be using a development enviroment for cross compiling like the Ridgerun SDK, you can download the eval version here:

http://developer.ridgerun.com/wiki/index.php?title=RidgeRun_Professional_SDK_Irazu_for_Nvidia_TegraX1_-_Release_Notes

The eval version doesn’t have the Qt5 package and is based on an on older version of JetPack , so you will have integrate that package into $DEVDIR/fs/apps (you can check other packages there as an example on how to integrate a package , is pretty straight forward since is Makefile based).