Need example Makefile of cross-comple for NX target

Hi, I want to setup a Linux host for cross-compling applications running on NX target. First I try to build an application to test the tegra watchdog timer /dev/watchdog0. I got the sample application code from this link:

Then I follow the instruction from

to create a Makefile to cross compile this application. But I always got “/bin/sh: 1: Syntax error: “(” unexpected”.

Can anyone provide an example Makefile for me?

My build host is a x86 running Ubuntu 18.04. The target is a Jetson_Xavier NX devkit (Linux 4.9.140-tegra).
I installed JetPack_4.4, Tegra_Linux_Sample-Root-Filesystem_R32.4.2_aarch64, and gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu on my host already.

Thank you very much.

This is my Makefile in case that anyone can correct it:

SYSROOT=/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs
CROSS_ROOT=/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu
CROSS_COMPLE=${CROSS_ROOT}/bin
CC=${CROSS_COMPLE}/aarch64-linux-gnu-gcc
CXX=${CROSS_COMPLE}/aarch64-linux-gnu-g++

LD=${CROSS_COMPLE}/aarch64-linux-gnu-ld
AR=${CROSS_COMPLE}/aarch64-linux-gnu-ar
AS=${CROSS_COMPLE}/aarch64-linux-gnu-as
RANLIB=${CROSS_COMPLE}/aarch64-linux-gnu-ranlib
#NVCC=/usr/local/cuda/nvcc

CFLAGS:= -Wall -Werror -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations
CFLAGS+= -I$(SYSROOT)/usr/include/aarch64-linux-gnu -I$(SYSROOT)/usr/include
#CFLAGS+= -I/usr/local/cuda/target/aarch64-linux/include

LIBS:= -L$(CROSS_ROOT)/aarch64-linux-gnu/lib64 -L$(CROSS_Root)/aarch64-linux-gnu/libc/usr/lib $(CROSS_Root)/aarch64-linux-gnu/libc/usr/lib/libc_nonshared.a -L$(SYSROOT)/usr/lib/aarch64-linux-gnu

LFLAGS:= -Wl,–start-group $(LIBS) -Wl,–end-group

SRC_FILES:= tegra_wdt.c

TARGET:= wdt_demo

all: $(TARGET)

$(TARGET) : $(SRC_FILES)
$(CC) -o $@ $^ $(CFLAGS) $(LFLAGS)

clean:
rm -rf *.o $(TARGET)

1 Like

I found the extra “(” in Makefile. Then expanded the variables in LIBS:= line. Now make runs. But I got implicit declaration for inctl(), write(), and close(). Which header should I include in the $SYSROOT path for those file operations ? There are multiple fcntl.h and ioctl.h files in the $SYSROOT tree. What is the CFLAG -I path for the system calls?

I included $(SYSROOT)/usr/include/sys/ioctl.h and $(SYSROOT)/usr/include/unistd.h, which solved the compile problem. But the program results in Segmentation fault. Does this related to wrong build setups?

Even I trim the program to an empty main(), it still results in :Segmentation fault". So it must be in the cross build setup. Can anyone provide some help?
Thanks.

I assume you are executing the program on the Jetson. If your program name is “hello”, what do you see from “file hello”? What do you see from “ldd hello”?

@linuxdev, this is the output:

sx-desktop:~$ file wdt_demo
wdt_demo: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=76057ea3ae079d58c54c8e6c4c88c8c00ba53751, with debug_info, not stripped

sx-desktop:~$ ldd wdt_demo
linux-vdso.so.1 (0x0000007f7d08d000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f7ced7000)
/lib/ld-linux-aarch64.so.1 (0x0000007f7d062000)

Thanks.

That shows the code is basically the right architecture for use on the Jetson.

Have you ever used gdb for debugging?

Yes. I can download gdb to the NX. is that what you want to me to do?

One more question. In my Makefile, the libc is supposed to be the libc_nonshared.a . But from ldd output I see the shared libc is linked. Is that a problem?

Thanks.

In a normal dynamically linked user space program you would use a “libc” with a “.so” in its filename extension. The “.a” is a statically linked special version, usually reserved for something needing some form of portability or running on bare metal. I would be very careful to not use libc_nonshared.a unless you have a good reason for it. A shared libc with linkage is normal.

There is a bit of a learning curve to gdb, but it isn’t too bad. You could start your program via:

gdb wdt_demo
r

The “gdb wdt_demo” loads the program into the debugger, then the “r” tells it to start running. It’ll either run to completion, or else fail somewhere.

If it fails, then you can see what the call stack is with a “backtrace” command:
bt

You could mouse copy and paste that as a reference.

Then, if you want to see the code source at the current point of failure, “list” the code:
l
…which would be useful as well. “q” is to quit out.

@linuxdev, I collected the following information from gdb. Looks like I missed some dependent files.

sx-desktop:~$ gdb ./wdt_demo
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.

This GDB was configured as “aarch64-linux-gnu”.

Reading symbols from ./wdt_demo…done.
(gdb) r
Starting program: /home/sxu/wdt_demo

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) by
Undefined command: “by”. Try “help”.
(gdb) bt
#0 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) l
1 …/sysdeps/aarch64/crti.S: No such file or directory.
(gdb)

I added -g in CFLAGS and copied the source file to the target. Re-run the gdb again, collected the following. The crti.S error is gone.

(gdb) b main
Breakpoint 1 at 0x884: file tegra_wdt.c, line 66.
(gdb) r
Starting program: /home/procerus/wdt_demo

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) l
warning: Source file is more recent than executable.
52 }
53
54 int main(int argc, char *argv)
55 {
56 int fd; /
File handler for watchdog /
57
58 int interval; /
Watchdog timeout interval (in secs) /
59 int bootstatus; /
Wathdog last boot status */
60 char dev; / Watchdog default device file */
61
(gdb) bt
#0 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

I compiled the same program on NX and it runs ok. So I still think that the cross-compile setup has problems.

This makes me wonder if the backtrace is valid:

This says it is a memory access error:

This makes me wonder about what you mentioned, that perhaps compile was somehow wrong:

…more importantly, that the compile may have linked against something which is different on the host system versus the target system. You were correct to add the “-g” compile option, which I had failed to mention.

Are you able to post the code up through the main() function?

If you are ambitious, then you might want to run this under strace to see if something obvious shows up. strace output is quite long, but errors before a crash will typically be in the last 100 lines or so. You could create a trace log like this (assuming your program name is “hello”):
strace -oLogTrace.txt hello

One comparison I would check is if the program, when built directly on a Jetson, does work, just to be sure it isn’t the program design itself.

@linuxdev, the main() function has just one line: “return 0;” The program was originally downloaded from the link stated in my first post.
The strace log has three lines:

execve(“./wdt_demo”, [“./wdt_demo”], 0x7fd5654908 /* 20 vars */) = 0
— SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} —
+++ killed by SIGSEGV (core dumped) +++

BTW, which libc should I link in the sample JetPack rootfs?

Thanks.

Can you compile and build your hello program on the Jetson? Then run the command which shows where it is linked:
ldd hello

Instead of guessing where it would be linked, we can just ask the Jetson itself where it would link. Then on the host PC sysroot it would be a mirror of that. It seems like the SIGSEGV is probably related to that linkage, but the data is insufficient to say exactly why.

On NX target, I build the application wdt_demo, it runs correctly:

sxu-desktop:~/app$ ldd ./wdt_demo
linux-vdso.so.1 (0x0000007fa8728000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fa8571000)
/lib/ld-linux-aarch64.so.1 (0x0000007fa86fd000)

sxu-desktop:~/app$ file ./wdt_demo
./wdt_demo: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-, for GNU/Linux 3.7.0, BuildID[sha1]=982f9edbcbf663d11734f8f42b22b831bc9141ff, with debug_info, not stripped

With both the natively compiled and cross compiled version on the NX, we need to compare (side by side) the “ldd” results. I see the results for the working version, but what is the result from the ldd of the cross compiled binary? The difference is likely the problem.

Those information were posted on Oct. 20, the number 6 post of this thread. I post them here again.
Thanks.

sxu-desktop:~$ ldd ./wdt_demo
linux-vdso.so.1 (0x0000007fa8dc4000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fa8c0e000)
/lib/ld-linux-aarch64.so.1 (0x0000007fa8d99000)

sxu-desktop:~$ file ./wdt_demo
./wdt_demo: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=8dd1e118634f0206c316a798a2dbcf75107e7fc2, with debug_info, not stripped

Which one is which? (note: probably the first one is working, it lists a libc.so.6 and an offset to where it linked)