AGX Xavier: Start binary file which is not a Linux kernel

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hello list,
I try to boot an ELF file or a binary file which is not a Linux kernel from cboot.

I achieved to boot a non-signed Linux kernel compiled by myself:

  • compile the l4kernel (tag: tegra-l4t-r32.5.1)
  • modify the kernel image header with mkbootimg to add ANDROID magic string
  • remove CONFIG_ENABLE_SECURE_BOOT from l4t.mk located in cboot directory (sha256sum cboot_src_t19x.tbz2 : 851a2c3b3dcfc7ed718050d55ce5941b2385e81bdc6d2d4cb6f324606dd5efe2)
  • apply the modification related to CBO_DT_SIZE in tegrabl_cbo.c to match cbo partition size
  • recompile cboot and flash it
  • modify cbo.dts to use TFTP boot
  • create cbo.dtb and flash it

My purpose is to start a simple ELF file which is is not a Linux kernel and if
possible remove the usage of mkbootimg to add the expected “android” header…

Do you already experiment this setting with cboot?
Can you provide some pointers (documentation or forum’s pages) or advices to
fulfill this task?

Regards.

2

This won’t be particularly useful, but consider that bootloaders and kernels are standalone bare metal programs, and do not have the usual support for local variables without manually working around it, and do not work with any kind of linking to other programs. If your ELF program is a standalone bare metal program, then you should be able to make it work; if not, then you’ll need to rewrite it to provide all of the things a user space environment would have due to the kernel. If it turns out that you are running a standalone bare metal kernel (not necessarily Linux), then you are in luck.

3

Hello linuxdev,
Hmm it depends (: (maybe i don’t get your point)
Suppose you want to start a binary that don’t need DTB and as its own header which could be different from Linux kernel.
I can give you 2 simple examples:

  • u-boot: i can imagine from cboot to use u-boot like TX1 or TX2 boot. It provides more flexibility to start something different of Linux kernel (boot ELF file, boot RAW binary file, …)
  • hypervisor which don’t need DTB but just need to run after the bootloader as a type 1 hypervisor.

From my current understanding, i don’t see any documentation that explain how to boot a “simple” binary with cboot.

Documentation, howto and advices are welcome.
Thanks.

4

Bootloaders and hypervisors are “bare metal” applications. Being ELF format does not guarantee bare metal.

To illustrate, if using C, then you could have automatic allocation of memory for a local variable:

void my_func()
{
   int foo;  // The operating system performs the allocation for you.

The above requires user space and an operating system. The above is not compatible with a bare metal environment. One would have to manually allocate memory for “int foo” in the above if you are running this bare metal…rephrased, anything you run directly without the operating system behind it gives up all of those nice services you get when running in an operating system. An ELF format does not mean it is coded for a bare metal environment. You might need to rewrite that code…all memory allocations would need to be manually added, no library linking would be available unless you code your own linker, so on. Even function “main()” will not work on bare metal…mostly you’d place some code somewhere and tell the interrupt handler to go to that address after the correct interrupt is detected.

The Linux kernel itself is bare metal. U-Boot is a bare metal application. Hypervisors are reached via an interrupt handler. Does your ELF program run purely from assembler, or does it have “main()”? Does your ELF program link to anything? Does your ELF program provide all of its own memory allocation, even for local variables?

5

Hello linuxdev,
My application is a bare metal application (hypervisor type 1) which is executed properly by u-boot in other configuration (TX1 for example).

By the way i create my binary image and run mkbootimg to comply to cboot expectation (especially tegrabl_verify_boot_img_hdr).

During execution i get the following output:
8—
[…]
[0002.166] I> Load in CBoot Boot Options partition and parse it
[0002.177] I> boot-order :-
[0002.177] I> 1.net
[0002.177] I> 2.sd
[0002.177] I> 3.usb
[0002.178] I> 4.nvme
[0002.180] I> 5.emmc
[0002.182] I> tftp-server-ip: 192.168.0.1
[0002.186] I> static-ip: 192.168.0.10
[0002.190] I> ip-netmask: 255.255.255.0
[0002.193] I> ip-gateway: 192.168.0.1
[0002.197] E> Failed to parse GUID
[0002.200] I> Hit any key to stop autoboot: 4 3 2 1
[0004.207] initializing target
[0004.207] calling apps_init()
[0004.208] starting app kernel_boot_app
[0004.227] I> found decompressor handler: lz4-legacy
[0004.228] I> decompressing BMP blob …
[0004.239] I> Kernel type = Normal
[0004.240] I> Loading kernel-bootctrl from partition
[0004.240] I> Loading partition kernel-bootctrl at 0xa42c0000 from device(0x1)
[0004.247] W> tegrabl_get_kernel_bootctrl: magic number(0x00000000) is invalid
[0004.247] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0004.248] I> ########## Net boot ##########
[0004.248] I> EQoS: Init
[0006.252] I> Wait till auto-calibration completes…
[0006.253] I> Configuring PHY
[0006.313] I> Start auto-negotiation
[0006.314] I> Wait till it completes…
[0009.315] I> MAC addr 00:04:4b:cb:9f:3d
[0009.315] I> Configure Static IP …
[0009.315] I> netif status changed 192.168.0.10
[0009.316] I> netif status changed 192.168.0.10
[0009.316] I> Our IP: 192.168.0.10
[0009.316] TFTP Client: Init
[0009.317] TFTP Client: Server IP: 192.168.0.1
[0009.317] TFTP Client: Send RRQ, file: jetson.dtb
[0014.321] TFTP Client: Connection failed
[0014.331] TFTP Client: Send RRQ, file: jetson.dtb

[0014.399] TFTP Client: Last packet received
[0014.400] TFTP Client: Send RRQ, file: boot.img
############
[0014.754] TFTP Client: Last packet received
[0014.755] I> netif status changed 192.168.0.10
[0014.756] I> Validate kernel …
[0014.756] I> Validate kernel-dtb …
[0014.756] I> Checking boot.img header magic … [0014.756] I> [OK]
[0014.757] I> Kernel hdr @0xa42c0000
[0014.757] I> Kernel dtb @0x90000000
[0014.757] I> decompressor handler not found
[0014.761] I> Copying kernel image (1610664 bytes) from 0xa42c0800 to 0x80080000 … [0014.769] I> Done
[0014.770] I> Move ramdisk (len: 0) from 0xa444a000 to 0x91000000
[0014.776] I> Updated bpmp info to DTB
[0014.780] I> Ramdisk: Base: 0x91000000; Size: 0x0
[0014.784] I> Updated initrd info to DTB
[0014.787] W> WARN: Fail to override “console=none” in commandline
[0014.793] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0014.801] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0014.808] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0014.815] W> No valid slot number is found in scratch register
[0014.821] W> Return default slot: _a
[0014.824] I> Active slot suffix:
[0014.827] I> add_boot_slot_suffix: slot_suffix =
[0014.832] I> Linux Cmdline: console=ttyTCU0,115200 video=tegrafb no_console_suspend=1 earlycon=tegra_comb_uart,mmio32,0x0c168
000 gpt usbcore.old_scheme_first=1 tegraid=19.1.2.0.0 maxcpus=8 boot.slot_suffix= boot.ratchetvalues=0.4.2 vpr_resize sdhci_te
gra.en_boot_part_access=1
[0014.857] I> Updated bootarg info to DTB
[0014.860] W> MAC addr invalid!
[0014.863] E> Failed to get WIFI MAC address
[0014.867] W> MAC addr invalid!
[0014.870] E> Failed to get Bluetooth MAC address
[0014.875] I> eeprom_get_mac_addr: MAC (type: 2): 00:04:4b:cb:9f:3d
[0014.881] E> “plugin-manager” doesn’t exist, creating
[0014.886] I> Adding /chosen/plugin-manager/cvm
[0014.890] E> “chip-id” doesn’t exist, creating
[0014.894] I> Adding /chosen/plugin-manager/chip-id
[0014.899] E> “configs” doesn’t exist, creating
[0014.903] I> Adding /chosen/plugin-manager/configs
[0014.908] E> “ids” doesn’t exist, creating
[0014.912] I> Adding /chosen/plugin-manager/ids
[0014.916] E> “odm-data” doesn’t exist, creating
[0014.920] I> Adding /chosen/plugin-manager/odm-data
[0014.928] E> “memory” doesn’t exist, creating
[0014.930] I> [0] START: 0x80000000, END: 0xac000000
[0014.934] I> [1] START: 0xac004000, END: 0xf0ad0000
[0014.939] I> [2] START: 0xf0adc000, END: 0xf0ae0000
[0014.943] I> dram_block larger than 80000000
[0014.947] I> [3] START: 0x100000000, END: 0x480000000
[0014.952] I> added [base:0x80000000, size:0x2c000000] to /memory
[0014.958] I> added [base:0xac200000, size:0x44800000] to /memory
[0014.964] I> added [base:0x100000000, size:0x380000000] to /memory
[0014.971] I> Updated memory info to DTB
[0014.974] E> add_disp_param: failed to get display params for du=0
[0014.980] E> “reset” doesn’t exist, creating
[0014.985] I> NVG: Logical CPU: 0; MPIDR: 0x80000000
[0014.989] I> NVG: Logical CPU: 1; MPIDR: 0x80000001
[0014.993] I> NVG: Logical CPU: 2; MPIDR: 0x80000100
[0014.998] I> NVG: Logical CPU: 3; MPIDR: 0x80000101
[0015.003] I> NVG: Logical CPU: 4; MPIDR: 0x80000200
[0015.008] I> NVG: Logical CPU: 5; MPIDR: 0x80000201
[0015.012] I> NVG: Logical CPU: 6; MPIDR: 0x80000300
[0015.017] I> NVG: Logical CPU: 7; MPIDR: 0x80000301
[0015.023] E> “misc-data” doesn’t exist, creating
[0015.026] I> Boot-device: eMMC
[0015.029] I> Add boot-sdmmc to plugin-manager/misc-data
[0015.034] I> Add storage-sdmmc to plugin-manager/misc-data
[0015.040] W> Unknown storage device
[0015.043] I> Add serial number:0423418004115 as DT property
[0015.050] I> Plugin-manager override starting
[0015.053] I> node /plugin-manager/fragement-tegra-wdt-en matches
[0015.060] I> node /plugin-manager/fragement-soft-wdt matches
[0015.068] I> node /plugin-manager/fragment-pcie-c5-rp matches
[0015.072] I> node /plugin-manager/fragment-tegra-ufs-lane10 matches
[0015.083] I> Disable plugin-manager status in FDT
[0015.084] I> Plugin-manager override finished successfully
[0015.085] I> tegrabl_load_kernel_and_dtb: Done
[0015.090] E> tegrabl_display_clear: display is not initialized
[0015.095] W> Boot logo display failed…
[0015.099] I> Kernel EP: 0x80080000, DTB: 0x90000000
[0015.104]
[0015.105] -----------------------------------------------
[0015.110] Synchronous Exception: UNKNOWN EXCEPTION
[0015.115] -----------------------------------------------
[0015.120]
[0015.121] ESR 0x2000000: ec 0x0, il 0x1, iss 0x0
[0015.126] -----------------------------------------------
[0015.131] [Stack Trace]
[0015.133]
[0015.135] => pc:0x80080000, sp:0xA0689740
[0015.138] => pc:0xA0603BEC, sp:0xA0689970
[0015.142] => pc:0xA0603C00, sp:0xA06899E0
[0015.146] => pc:0xA0603884, sp:0xA0689A20
[0015.150] => pc:0xA0602DF8, sp:0xA0689A30
[0015.154] => pc:0xA0602DCC, sp:0xA0689A40
[0015.158] -----------------------------------------------
[0015.163] iframe 0xa0689650:
[0015.166] x0 0x 90000000 x1 0x 0 x2 0x 0 x3 0x 0
[0015.175] x4 0x 80080000 x5 0x 20 x6 0x b200123 x7 0x ffffffc0
[0015.184] x8 0x 0 x9 0xffffffffffffffff x10 0x 6 x11 0x 2
[0015.193] x12 0x 1 x13 0x 40 x14 0x 1 x15 0x 2c0
[0015.202] x16 0x 0 x17 0x 0 x18 0x 0 x19 0x 0
[0015.211] x20 0x a06899f0 x21 0x 0 x22 0x 0 x23 0x 0
[0015.220] x24 0x 0 x25 0x 0 x26 0x 0 x27 0x 0
[0015.229] x28 0x 0 x29 0x a0689970 lr 0x a0603ba0 sp 0x a0689740
[0015.238] elr 0x 80080000
[0015.241] spsr 0x 400003c9
[0015.245] -----------------------------------------------
[0015.250] panic (caller 0xa0601238): die
[0015.254] HALT: spinning forever…
—8

The following line draws my attention
8—
[0015.099] I> Kernel EP: 0x80080000, DTB: 0x90000000
—8

So the expected entry point virtual address is 0x80080000 which is not my case…
When i dig a littel bit into cboot source the function tegrabl_get_kernel_load_addr retrieves the entry point from the boot_params variable.

How can i update boot_params properly to modify my entry point?

Thanks.

6

The above should not matter if there is a fallback device. GUID is just one way to find a partition for mount.

Don’t know if it matters, but the above would essentially be passed as an argument to your bare metal application when execution transfers to it upon completion of CBoot.

The above makes me wonder if (A) your code uses the missing storage device, or if (B) you don’t use that device but attempts to read that device causes a failure.

The above seems to be where problems start, and graphics or framebuffer are being attempted. Does your bare metal application use graphics at all? If so, perhaps you could try a version with no graphics or framebuffer just to see if it changes the failure.

Overall there isn’t enough to be specific for debug. I’ve not studied the CBoot code enough to answer the question on address, but this could certainly cause such an error, and the error message might be completely unexpected if it is executing code at the wrong address. I couldn’t say from this if that is the actual case, but it is reasonable to check.

Would someone else be able to say how to change the kernel load address from CBoot, such that it uses 0x8008000 instead of 0x80000000 (or some other address which might have been used in CBoot for the Linux kernel)?

7

Hi list,

No update concerning this issue?
I found something similar in this thread QNX &.x support / QNX BSP
but this is not a “satisfying” answer.
It seems that QNX officially support the board. See QNX Board Support Packages - Hardware | BlackBerry QNX.

Regards.

10

Hi list,
I fix the issue by:

  • let the mmu region to host the code by removing MMU_FLAG_EXECUTE_NOT
  • let the mmu as configured before booting the binary by commenting platform_uninit
    Thanks for your support.