Need example Makefile of cross-comple for NX target

The first one posted on 23rd is working version. Both of two set output are collected from a NX target.
Thanks.

Therefore, since this is the difference:
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fa8c0e000)
…the missing link is “libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6”. Your cross compile needs to be adjusted to use this.

If you take the failing wdt-demo, and run the cross arch ldd on it from the host PC, does it say anything about libc.so.6?

Do you mean I need to change the LIBS path in my Makefile, posted on Oct. 19th in this thread.
Thanks.

I’m not sure of the exact place to change this, but when building on the host PC it is failing to provide the ability for the linker to use “/lib/aarch64-linux-gnu/libc.so.6”. The library exists on the Jetson at that location. The library probably exists on the cross compile, but not in such a way that the resulting binary is able to use the linker to find that file.

If you use the cross tool version of “ldd” on your host PC to see what the binary links to, what do you see? In the same way that you put a copy of both versions of the binary on your Jetson and compared the ldd results, you could do this on your PC as well (though you’d have to use the cross ldd instead of native ldd).

On the host box, I used the aarch64-linux-gnu-ld command since there is no -ldd in the tool chain to get the following information:

~$ ~/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld ./wdt_demo.bad
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: libc.so.6, needed by ./wdt_demo.bad, not found (try using -rpath or -rpath-link)
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: cannot find entry symbol _start; not setting start address
./wdt_stuartx@stuartx-HP-ZBook-17-G2:~/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/sources/app$ ~/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld ./wdt_demo
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: libc.so.6, needed by ./wdt_demo.bad, not found (try using -rpath or -rpath-link)
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: cannot find entry symbol _start; not setting start address
./wdt_demo.bad: undefined reference to fwrite@GLIBC_2.17' ./wdt_demo.bad: undefined reference to fprintf@GLIBC_2.17’
./wdt_demo.bad: undefined reference to exit@GLIBC_2.17' demo.bad: undefined reference to fwrite@GLIBC_2.17’
./wdt_demo.bad: undefined reference to fprintf@GLIBC_2.17' ./wdt_demo.bad: undefined reference to exit@GLIBC_2.17’

Then, I modified the LIBS flags in the Makefile. Replaced all -L to -Wl,rpath:

SYSROOT=/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs
CROSS_ROOT=/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu

LIBS:= -Wl,-rpath,$(CROSS_ROOT)/aarch64-linux-gnu/lib64 -Wl,-rpath,$(CROSS_ROOT)/aarch64-linux-gnu/libc/usr/lib -Wl,-rpath,$(SYSROOT)/usr/lib/aarch64-linux-gnu -Wl,-rpath,$(SYSROOT)/lib/aarch64-linux-gnu

I recompiled the program, and get the following:

$ ~/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld ./wdt_demo
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: libc.so.6, needed by ./wdt_demo, not found (try using -rpath or -rpath-link)
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: cannot find entry symbol _start; not setting start address
./wdt_demo: undefined reference to fwrite@GLIBC_2.17' ./wdt_demo: undefined reference to close@GLIBC_2.17’
./wdt_demo: undefined reference to getline@GLIBC_2.17' ./wdt_demo: undefined reference to abort@GLIBC_2.17’
./wdt_demo: undefined reference to ioctl@GLIBC_2.17' ./wdt_demo: undefined reference to stderr@GLIBC_2.17’
./wdt_demo: undefined reference to malloc@GLIBC_2.17' ./wdt_demo: undefined reference to atoi@GLIBC_2.17’
./wdt_demo: undefined reference to stdout@GLIBC_2.17' ./wdt_demo: undefined reference to write@GLIBC_2.17’
./wdt_demo: undefined reference to open@GLIBC_2.17' ./wdt_demo: undefined reference to optarg@GLIBC_2.17’
./wdt_demo: undefined reference to fprintf@GLIBC_2.17' ./wdt_demo: undefined reference to strerror@GLIBC_2.17’
./wdt_demo: undefined reference to getopt_long@GLIBC_2.17' ./wdt_demo: undefined reference to stdin@GLIBC_2.17’
./wdt_demo: undefined reference to exit@GLIBC_2.17' ./wdt_demo: undefined reference to __errno_location@GLIBC_2.17’
$ ~/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld ./wdt_demo

/home/stuartx/
l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: libc.so.6, needed by ./wdt_demo, not found (try using -rpath or -rpath-link)
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-ld: warning: cannot find entry symbol _start; not setting start address
./wdt_demo: undefined reference to fwrite@GLIBC_2.17' ./wdt_demo: undefined reference to close@GLIBC_2.17’
./wdt_demo: undefined reference to getline@GLIBC_2.17' ./wdt_demo: undefined reference to abort@GLIBC_2.17’
./wdt_demo: undefined reference to ioctl@GLIBC_2.17' ./wdt_demo: undefined reference to stderr@GLIBC_2.17’
./wdt_demo: undefined reference to malloc@GLIBC_2.17' ./wdt_demo: undefined reference to atoi@GLIBC_2.17’
./wdt_demo: undefined reference to stdout@GLIBC_2.17' ./wdt_demo: undefined reference to write@GLIBC_2.17’
./wdt_demo: undefined reference to open@GLIBC_2.17' ./wdt_demo: undefined reference to optarg@GLIBC_2.17’
./wdt_demo: undefined reference to fprintf@GLIBC_2.17' ./wdt_demo: undefined reference to strerror@GLIBC_2.17’
./wdt_demo: undefined reference to getopt_long@GLIBC_2.17' ./wdt_demo: undefined reference to stdin@GLIBC_2.17’
./wdt_demo: undefined reference to exit@GLIBC_2.17' ./wdt_demo: undefined reference to __errno_location@GLIBC_2.17’

Using the ldd from the host, I get:
$ ldd ./wdt_demo.bad
not a dynamic executable

On the target:
$ ldd ./wdt_demo
linux-vdso.so.1 (0x0000007fa24d2000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fa231b000)
/lib/ld-linux-aarch64.so.1 (0x0000007fa24a7000)

The above is saying there was no libc.so.6 available for runtime, and this would also be a problem during the linking stage of build. Here is an excerpt of setup from a previous post which is related to this:

If you check for the existence of libc.so.6 here, then it was a case of the linker path not having been set during build to include this directory:

/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs/lib/aarch64-linux-gnu/

Notice how I appended “lib/aarch64-linux-gnu/”, as this directory should contain libc.so.6 in the rootfs content.

You could also check on your host PC and see if a libc.so.6 exists here:
/lib/aarch64-linux-gnu/
(package based cross tools for a sysroot typically add to the same “/lib” as the main computer, but named after the foreign architecture)

Whenever a linker tries to work, it has a predefined default linker path. For a native package that is set up automatically. For cross linking, you need to specify the path, or else have it in a “standardized” location, e.g., “/lib/aarch64-linux-gnu`”. Your build process never set up the linker path.

The comments in the ld command about using “-rpath” should be ignored (unless you want to guarantee your specific file is used in a custom non-standard path).

When you look for libc.so.6 with the above information, where do you find this? Is it in your rootfs path? Is it in “/lib/aarch64-linux-gnu”? Are you set up to pass information to the linker about where to search? In a foreign compile you would probably pass directories to ld via the “-L searchdir” argument.

NOTE: I tend to try to use standard paths, and then mount a clone of the target system, followed by placing symbolic links into those standard directories which point at the clone mount.

The directory

/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs/lib/aarch64-linux-gnu/

is in the link path. And I checked that the libc.so.6 is in the directory. I also to tried to put a symbol-ink in the host /lib/aarch64-linux-gun/, the program still does not work. The following is my Makefile:

SYSROOT=/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs
CROSS_ROOT=/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu
CROSS_COMPLE=${CROSS_ROOT}/bin
CC=${CROSS_COMPLE}/aarch64-linux-gnu-gcc
CXX=${CROSS_COMPLE}/aarch64-linux-gnu-g++
LD=${CROSS_COMPLE}/aarch64-linux-gnu-ld
AR=${CROSS_COMPLE}/aarch64-linux-gnu-ar
AS=${CROSS_COMPLE}/aarch64-linux-gnu-as
RANLIB=${CROSS_COMPLE}/aarch64-linux-gnu-ranlib
#NVCC=/usr/local/cuda/nvcc


#$(info **************** start compile )

CFLAGS:= -Wall -shared -fPIC -Wno-error=deprecated-declarations
CFLAGS+= -I$(SYSROOT)/usr/include/aarch64-linux-gnu -I$(SYSROOT)/usr/include
CFLAGS+= -g
#CFLAGS+= -I/usr/local/cuda/target/aarch64-linux/include


LIBS:= -L$(CROSS_ROOT)/aarch64-linux-gnu/lib64/ -L$(CROSS_ROOT)/aarch64-linux-gnu/libc/usr/lib/ -L$(SYSROOT)/usr/lib/aarch64-linux-gnu/ -L$(SYSROOT)/lib/aarch64-linux-gnu/
#LIBS:= -Wl,-rpath,$(SYSROOT)/

#LIBS:= -L/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs/lib/aarch64-linux-gnu

LFLAGS:= -Wl,--start-group $(LIBS) -Wl,--end-group


SRC_FILES:= tegra_wdt.c

TARGET:= wdt_demo

all: $(TARGET)

$(TARGET) : $(SRC_FILES)
        $(CC) -o $@ $^ $(CFLAGS) $(LFLAGS)


clean:
        rm -rf *.o $(TARGET)

I have this symbolic link:

$ ls -l /lib/aarch64-linux-gnu
lrwxrwxrwx 1 root root 125 Oct 28 14:23 /lib/aarch64-linux-gnu -> /home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs/lib/aarch64-linux-gnu/

This is usually a symbolic link. Is the link valid? If using an absolute path, then I would expect it to fail. If using a relative path, then ti should work.

Mostly from what I can see it should work. The trick is that when cross compiling, if you have a cross linker with a default path which works, then it should be able to link to that particular libc.so.6. I’m not sure if there is a “cross” version of “ldconfig -p”, but it might be that passing the argument during build is correct, but if the linker at runtime does not have the correct default path, then it will still appear as if it failed (the link would have possibly taken place correctly, but running ld later might not have the same result if ld itself does not know about that libc.so.6 location).

The libc.so.6 is a symbolic link. Both of these files are exactly same as those on the target

-rwxr-xr-x 1 root root 1341080 Apr 16  2018 libc-2.27.so
lrwxrwxrwx 1 root root      12 Apr 16  2018 libc.so.6 -> libc-2.27.so

So we know the file used during compiler link stage is the right file.

I am struggling a bit here because I’m used to being able to use ldd, and running ld as a substitute in the cross environment is not providing all of the same information. In theory, you could normally run “ldconfig -p” to list linker default details, but this too is probably missing from your cross environment. Is there a possibility that you can build the simple test case again, but with verbose options?

Verbose will provide a lot of details, but if the code being compiled is just the basic main() with a return 0;, then the output should be tolerable. What you would need to do is pass “-v” to gcc, e.g.:
gcc -v ...the C file...

I added -v in Makefile:

$(TARGET) : $(SRC_FILES)
        $(CC) -v -o $@ $^ $(CFLAGS) $(LFLAGS)

In the make output, I see some “ignore nonexistent directory” lines. I am not sure why the compiler looked for those directories. I manually checked all the compiler include and lib link paths, All of them are valid. The make output is uploaded.

Thanks.make_output.txt (16.4 KB)

Those directories are from your toolchain:

ignoring nonexistent directory "/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/../aarch64-linux-gnu/libc/usr/local/include/aarch64-linux-gnu"
ignoring nonexistent directory "/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/../aarch64-linux-gnu/libc/usr/local/include"
ignoring nonexistent directory "/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/../aarch64-linux-gnu/libc/usr/include/aarch64-linux-gnu"

It looks like it is dropping out of a toolchain version-specific naming and going back in to a generic “aarch64-linux-gnu/” tree. It is important to know that the original location is for the binary toolchain, but the new location is not for tools…this is for the C library headers. I’m not sure if this is harmful or not, as there are probably other header “include” locations, and those locations might make available the headers you need. It does make me suspicious that something else is wrong in the same way, but I don’t know.

I see this:

LIBS:= -L$(CROSS_ROOT)/aarch64-linux-gnu/lib64/ -L$(CROSS_ROOT)/aarch64-linux-gnu/libc/usr/lib/ -L$(SYSROOT)/usr/lib/aarch64-linux-gnu/ -L$(SYSROOT)/lib/aarch64-linux-gnu/

…which has this excerpt:
-L$(SYSROOT)/lib/aarch64-linux-gnu/
…which says probably libc.so.6 is available. Can you verify that " -L$(SYSROOT)/lib/aarch64-linux-gnu/" contains libc.so.6?

Yes,

-L$(SYSROOT)/lib/aarch64-linux-gnu/

has libc in it:

stuartx@stuartx-HP-ZBook-17-G2:~/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs/lib/aarch64-linux-gnu$ ls -l libc*
-rwxr-xr-x 1 root root 1341080 Apr 16  2018 libc-2.27.so
lrwxrwxrwx 1 root root      18 Sep 24  2017 libcap-ng.so.0 -> libcap-ng.so.0.0.0
-rw-r--r-- 1 root root   18584 Sep 24  2017 libcap-ng.so.0.0.0
lrwxrwxrwx 1 root root      14 Nov 16  2017 libcap.so.2 -> libcap.so.2.25
-rw-r--r-- 1 root root   22600 Nov 16  2017 libcap.so.2.25
-rw-r--r-- 1 root root  186536 Apr 16  2018 libcidn-2.27.so
lrwxrwxrwx 1 root root      15 Apr 16  2018 libcidn.so.1 -> libcidn-2.27.so
lrwxrwxrwx 1 root root      17 Jan 22  2020 libcom_err.so.2 -> libcom_err.so.2.1
-rw-r--r-- 1 root root   14104 Jan 22  2020 libcom_err.so.2.1
-rw-r--r-- 1 root root   30880 Apr 16  2018 libcrypt-2.27.so
lrwxrwxrwx 1 root root      23 Aug 23  2018 libcryptsetup.so.12 -> libcryptsetup.so.12.2.0
-rw-r--r-- 1 root root  273072 Aug 23  2018 libcryptsetup.so.12.2.0
lrwxrwxrwx 1 root root      16 Apr 16  2018 libcrypt.so.1 -> libcrypt-2.27.so
lrwxrwxrwx 1 root root      12 Apr 16  2018 libc.so.6 -> libc-2.27.so

Would you attach your actual Makefile from whatever version is your most recent (presumably the one which produced the wdt_demo which did not link on the Jetson)? You can rename it with a “.txt” suffix and it should post. I’m going to need to actually compile to see what is going on. I created a stub “wdt_demo.c” which is just a main() with a “hello” type print, and adjusted for my Linaro 7.3, but I assume something is incomplete in the Makefile I am using since it says:
make: *** No rule to make target 'tegra_wdt.c', needed by 'wdt_demo'. Stop.

I am thinking that if I can compile this locally and work directly on it, then it should be easier to solve.

I am attaching two files. Makefile (Makefile.txt) and the code tegra_wdt.c (tegra_wdt.txt).
Makefile.txt (1.3 KB) tegra_wdt.txt (6.7 KB)

The make program on my host is:

stuartx@stuartx-HP-ZBook-17-G2:~/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/sources/app$ make -v
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Thanks.

I am finding that this build leaves part of the system saying “not a dynamic executable” once copied to the Jetson from the cross compile environment. I am still working on this, just letting you know I didn’t forget.

It looks like there are a lot of arguments being passed which are not needed for a normal executable program, e.g., for building libraries with dynamically loadable content (you don’t normally use this with an executable). I have not finished this yet, but I will show you something which will be of interest. First of all, I created “tegra_wdt.c” in a simplified manner:

/* Stub for main() */
#include <stdio.h>

int main()
{
	printf("hello\n");
	return 0;
}

I used the printf() because I wanted to know for certain the program is trying to link user space libraries.

I installed “CROSS_ROOT=/usr/local/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu” on my host PC (Fedora) via the NVIDIA provided precompiled recommended toolchain. My actual test system is a Jetson NX running L4T R4.3, whereas you reported using L4T R32.4.2. I believe this will not be any different, and that any differences or failures are unrelated to this release version.

The result of the Makefile build has issues which reproduce exactly as you described. The use of gdb and strace did not offer clues, but “ldd” showed that once the executable is copied to the Jetson “not a dynamic executable” despite the “file” command disagreeing. The key seems to be not “dynamic”. This seems to be a result of arguments passed during build being for a library and not an executable.

If I compile only the “tegra_wdt.o” with the Makefile (“make tegra_wdt.o”), this works. Note that no linking is performed yet.

If I then complete build of “wdt_demo” by linking the existing “tegra_wdt.o”, then I get the “not an executable” problem (as shown by ldd after copy to the Jetson).

If I instead build “wdt_demo” on command line with limited arguments, the program works, and ldd no longer complains “not an executable”. Here is an example of command line build once I already have “tegra_wdt.o” build, and I limit to purely linking (adjust for your file paths):

export SYSROOT=/home/stuartx/nvidia/nvidia_sdk/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/rootfs
/home/stuartx/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc  \
 --sysroot="${SYSROOT}" \
 -g tegra_wdt.o -o wdt_demo

Notice that I am typing this directly into a shell and this is why I “export”. I am also not exporting and substituting the actual “-gcc” binary.

Can you try this and verify that when you copy the program to your Jetson that it executes and prints the “hello”?

Yes. following your command line input, the program is cross built on the host and runs on NX. It works!

On the target, I run ldd for both the programs locally build and cross-build, the output information are different.
The wdt_demo built from NX taget has the following ldd output:

sxu-desktop:~/app$ ldd ./wdt_demo
linux-vdso.so.1 (0x0000007f86505000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f8634f000)
/lib/ld-linux-aarch64.so.1 (0x0000007f864da000)

The cross-build wdt_demo’s ldd output:

sxu-desktop:~/from_host$ ldd ./wdt_demo
linux-vdso.so.1 (0x0000007fa1d03000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fa1b5f000)
/lib/ld-linux-aarch64.so.1 (0x0000007fa1cd8000)

Thanks.

I did not go through the full Makefile edits needed, but what it comes down to is that you should limit library options to not be used on the executable. The main() symbol (and the symbols related to main()) seem to be incorrectly generated when you mix in library options. If you are actually compiling a library, then options like “-fPIC” would apply.

Try to separate the generation of the “.o” object files to one set of arguments, and make the linking of those “.o” files a separate step with separate options. If you create a library, then you need to also segregate the dynamic and static libraries to their specific arguments. Executable programs being linked are already mentioned, but to emphasize, the arguments for an executable will differ from the either static or dynamic libraries.

Sorry for the late response because I took off a couple of days.
Following your advice, I created 4 scenarios:

  1. separate compile and link, without the LFLAGS. It works.
tegra_wdt.o : tegra_wdt.c
        $(CC) -c tegra_wdt.c $(CFLAGS)

wdt_demo : tegra_wdt.o
       $(CC) -v -o wdt_demo tegra_wdt.o
  1. Separate the compile and link, with the LFLAGS. It works as well.
tegra_wdt.o : tegra_wdt.c
        $(CC) -c tegra_wdt.c $(CFLAGS)

wdt_demo : tegra_wdt.o
       $(CC) -v -o wdt_demo tegra_wdt.o $(LFLAGS) 
  1. AS I did before, put the compile and link in one line. It does not work no matter with or without the LFLGAS.
$(TARGET) : $(SRC_FILES)
#       $(CC) -v -o $@ $^ $(CFLAGS)
        $(CC) -v -o $@ $^ $(CFLAGS) $(LFLAGS)

So the LFLAGS is not the problem, though I don’t understand what is the root cause for the problemmake_work_noLflags.txt (8.5 KB) make_notWork_noLflags.txt (13.6 KB) . I am uploading two makeout.txt files. One is from the scenario 1 and the other from scenario 3. Both do not have the LFLAGS during build. These two files may provide a clue for the problem.