Kernel not booting when using dtb specified in extlinux.conf

Hello. I’m trying to boot my Jetson Tx2i via sdcard with a device tree specified in extlinux.conf, but the Kernel is not booting, and instead just hangs after the final Starting kernel ... message from u-boot.

To summarise my setup:

  • The Jetson was initially flashed via the standard Jetpack SDK build + flash process. This worked fine and I have been using this method for a while
  • I have recently tried to create a Yocto build for the Jetson following the process outlined here: Yocto Support for NVIDIA Jetson Platforms - Setting up Yocto.
  • Where I deviated from the above process was that I used the kirkstone-l4t-r32.7.x branch rather than the dunfell one.
  • This has also worked fine, and I can boot via my sdcards, using the images build via Yocto. For creating the SD cards, either the .wic output, or just dd’ing the .ext4 directory to a partition on the SD card works fine.
  • The above process however will just use the dtb contained in /dev/mmcblk0p28 as I did not change extlinux.conf

I am now trying to boot using a dtb contain in /boot. The only modification I’ve done to the above process is the following:

  • Place a dtb under /boot. This can be done by either mounting the sd card, or via the Yocto build process and adding kernel-devicetree to IMAGE_INSTALL. Either way the outcome is the same and the dtb generated during the Yocto build process is placed in /boot
  • Add an FDT entry in extlinux.conf to point to this file. This is done again via mounting an manual editing, or via adding UBOOT_EXTLINUX_FDT = "/boot/${KERNEL_DEVICETREE}" to my local.conf

Those are the only changes done, and rebooting and attempting to boot from the sd card will then yield the following, and no further output.

switch to partitions #0, OK
mmc2 is current device
Scanning mmc 2:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
228 bytes read in 91 ms (2 KiB/s)
1:	Poky (Yocto Project Reference Distro)
Retrieving file: /boot/extlinux/../initrd
2095170 bytes read in 359 ms (5.6 MiB/s)
Retrieving file: /boot/extlinux/../Image
38797320 bytes read in 4108 ms (9 MiB/s)
append: root=/dev/mmcblk1p1 rootwait rw rootfstype=ext4 console=ttyS2,115200n8 console=tty2 fbcon=map:0 net.ifnames=0 video=tegrafb no_console_suspend=1 earlycon=uart8250,mmio32,0xc280000 nvdumper_reserved=0x2372e0000 gpt usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.2031647.1 bl_prof_dataptr=0x10000@0x235840000 sdhci_tegra.en_boot_part_access=1  
Retrieving file: /boot/tegra186-quill-p3489-1000-a00-00-ucm1.dtb
524288 bytes read in 122 ms (4.1 MiB/s)
## Flattened Device Tree blob at 82400000
   Booting using the fdt blob at 0x82400000
   reserving fdt memory region: addr=80000000 size=10000
   Using Device Tree in place at 0000000082400000, end 0000000082482fff

Starting kernel ..

At this point I see no more output, nor can I ssh into the Jetson, so I’m assuming that the Kernel has panicked before it’s even had time to initialise serial output.

I considered that my device tree was broken, so I also tested it by pulling the device tree from the eMMC boot and using that, and that had the same effect. To do this I just used dtc on /sys/firmware/devicetree/base to generate the .dtb that was in use, and placed that under /boot, but again the above happened.

One thing to note is that I am not signing the dtb’s going into /boot, is this required?

Is there some step I’m missing here to allow me to boot via this method? To summarise:

  • Using Jetpack for the build/flash works
  • Using Yocto to create an SD card image and using the dtb in the eMMC partition works
  • Using the same SD card image except specifying the device tree via extlinux.conf fails

Any help is appreciated. Thanks

Hi,

Could we reproduce this issue without using Yocto? For example, creating a sdcard image by using only using files from jetpack and then test the same functionality.

Hi WayneWWW. Yeah I can try that actually. I have a second SD card in the carrier board which contains the image flashed via only the JetPack files, so I’ll mount that and add the device tree and changes to extlinux.conf to that.

Hmm how interesting that did work

Tegra186 (P2771-0000-500) # run bootcmd_mmc1
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
933 bytes read in 95 ms (8.8 KiB/s)
L4T boot options
1:	primary kernel
2:	primary-rw kernel
Enter choice: 1:	primary kernel
Retrieving file: /boot/initrd.img
20523800 bytes read in 2455 ms (8 MiB/s)
Retrieving file: /boot/Image
34441224 bytes read in 3756 ms (8.7 MiB/s)
append: root=/dev/mmcblk2p1 rootwait rw rootfstype=ext4 console=ttyS2,115200n8 console=tty2 fbcon=map:0 net.ifnames=0 video=tegrafb no_console_suspend=1 earlycon=uart8250,mmio32,0xc280000 nvdumper_reserved=0x2372e0000 gpt usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.2031647.1 bl_prof_dataptr=0x10000@0x235840000 sdhci_tegra.en_boot_part_access=1 ro overlay=tmpfs
Retrieving file: /boot/copy_of_dtb_from_jetpack.dtb
524288 bytes read in 212 ms (2.4 MiB/s)
## Flattened Device Tree blob at 82400000
   Booting using the fdt blob at 0x82400000
   Using Device Tree in place at 0000000082400000, end 0000000082482fff

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x100
...

It’s interesting that there’s no

reserving fdt memory region: addr=80000000 size=10000

Also interesting is that extlinux.conf doesn’t contain the FDT key/val pair once I boot from the sd card, but if I boot via eMMC and mount the sd card, I see the FDT key/val pair. So extlinux is getting overwritten during boot (which I believe I’ve read can happen?)

Perhaps it’s the initrd. I see in the eMMC and working SD card, extlinux.conf uses initrd.img, which is referenced via:

-rw-r--r--  1 root       root        5565090 Aug 30  2019 initrd
-rw-r--r--  1 root       root        5565090 Sep  5  2019 initrd.backup
lrwxrwxrwx  1 root       root             24 Aug 30  2019 initrd.img -> initrd.img-4.9.140-tegra
-rw-r--r--  1 root       root       20523800 Sep  5  2019 initrd.img-4.9.140-tegra

Meanwhile on the SD card created via Yocto, extlinux.conf just uses initrd. Perhaps this is related…

Ok so some sort of progress. I copied /boot/Image from the other sdcard containing the JetPack generated images onto the sdcard with the Yocto generate images, overwriting the /boot/Image there. Obviously this is not ‘correct’, but it was just for some extra information. I then added the FDT entry to point to the .dtb generated from the Yocto build.

This did sort of work though, and I got output up to here:

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x100
[    0.000000] Linux version 4.9.140-tegra (gcc version 7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) ) #4 SMP PREEMPT Fri Sep 20 12:06:58 BST 2019
[    0.000000] Boot CPU: AArch64 Processor [411fd073]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 16416,
[    0.000000] OF: fdt: - 80000000 ,  70000000
[    0.000000] OF: fdt: - f0200000 ,  145600000
[    0.000000] OF: fdt: - 235e00000 ,  200000
[    0.000000] OF: fdt: - 236600000 ,  200000
[    0.000000] OF: fdt: - 237000000 ,  200000
[    0.000000] earlycon: uart8250 at MMIO32 0x000000000c280000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb2_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb2_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fbc00000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 25 pages/cpu @ffffffc1b5f6a000 s61592 r8192 d32616 u102400
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1764408
[    0.000000] Kernel command line: root=/dev/mmcblk1p1 rootwait rw rootfstype=ext4 console=ttyS2,115200n8 console=tty2 fbcon=map:0 net.ifnames=0 video=tegrafb no_console_suspend=1 earlycon=uart8250,mmio32,0xc280000 nvdumper_reserved=0x2372e0000 gpt usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.2031647.1 bl_prof_dataptr=0x10000@0x235840000 sdhci_tegra.en_boot_part_access=1  
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 163840 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 262144 bytes
[    0.000000] early log buf free: 29392(89%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Memory: 6248448K/7170048K available (15230K kernel code, 2914K rwdata, 6876K rodata, 8576K init, 609K bss, 167936K reserved, 753664K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f60000   ( 15232 KB)
[    0.000000]     .rodata : 0xffffff8008f60000 - 0xffffff8009620000   (  6912 KB)
[    0.000000]       .init : 0xffffff8009620000 - 0xffffff8009e80000   (  8576 KB)
[    0.000000]       .data : 0xffffff8009e80000 - 0xffffff800a158808   (  2915 KB)
[    0.000000]        .bss : 0xffffff800a158808 - 0xffffff800a1f0dbc   (   610 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf06dc8000   (   109 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc1b7200000   (  7026 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=6.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=6
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000003] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.010021] Console: colour dummy device 80x25
[    0.014730] console [tty2] enabled
[    0.018320] bootconsole [uart8250] disabled

So to summarise:

  • With Yocto /boot/Image and JetPack dtb from eMMC partition. Boots
  • With JetPack generated /boot/Image and dtb from Yocto build. Partial boot
  • With Yocto /boot/Image and dtb from Yocto build. Doesn’t get beyond Starting kernel ...