Support Nano on OpenWRT

xback · June 29, 2022, 9:13am

Hi all,

I’ve been part of the OpenWRT dev team for a few years now.

As Nano seems to be declared EoL as far as upgrading goes, I would like to give this series extended life though supporting it in OpenWRT.

By “support” I really mean:

Being able to boot properly and support all board peripherals
Full GPU/CUDA support
Add support for Deepstream + YOLO which is a very popular combination

The initial focus would be to support Nano Developer board as a reference.
Depending on progress and hurdles, other models could follow (like TX2 etc)

Following advantages would arise from it:

A lot newer kernel supporting many modern add-on pheripherals and speedups
Hugely smaller image sizes
Modern gstreamer (currently 1.20.x is in use over there)
Upgrading could be examined using the OpenWRT sysupgrade utility
Community driven updates and maintenance for years to come

Logically, there would be input required from NVIDIA for this:

CUDA support with a minimum amount of dependencies which can be run on modern kernels
Minimalistic Drivers+Binaries for the GPU
…

I have experience with the concept of accelerated gstreaming using openwrt by means of the i.MX6 target which has hardware encoding support too requiring binaries.

I’m fully open to address any issues regarding disclosure by means of NDA’s.

Therefore, I would really like to kindly request the nvidia staff to seriously discuss this matter internally and provide feedback.
Feel free to PM me for personal details if there is any requirement for a conference call or official/legal paperwork.

Let’s make Jetson legendary while there is momentum … together :-)

kayccc · July 6, 2022, 3:30am

I have forwarded this inforamtion to our internal team to do the discussion.

xback · July 7, 2022, 3:07pm

Purely to inform you I’m already working on this in my staging tree

It already builds the kernel fine at this stage.

kayccc · July 8, 2022, 12:40am

Great! Thanks for your sharing!

xback · July 11, 2022, 12:12pm

Early test:

U-Boot 2020.04-g6b630d64fd (Feb 19 2021 - 08:37:46 -0800)

SoC: tegra210
Model: NVIDIA Jetson Nano Developer Kit
Board: NVIDIA P3450-0000
DRAM:  4 GiB
MMC:   sdhci@700b0000: 1, sdhci@700b0600: 0
Loading Environment from SPI Flash... SF: Detected mx25u3235f with page size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
835 bytes read in 27 ms (29.3 KiB/s)
L4T boot options
1:      primary kernel
2:      OpenWRT kernel
Enter choice: 2
2:      OpenWRT kernel
Retrieving file: /boot/initrd
7159624 bytes read in 333 ms (20.5 MiB/s)
Retrieving file: /boot/Image2
13357064 bytes read in 619 ms (20.6 MiB/s)
append: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 max0
## Flattened Device Tree blob at 83100000
   Booting using the fdt blob at 0x83100000
ERROR: reserving fdt memory region failed (addr=0 size=0)
ERROR: reserving fdt memory region failed (addr=0 size=0)
   Using Device Tree in place at 0000000083100000, end 000000008317e68d
copying carveout for /host1x@50000000/dc@54200000...
copying carveout for /host1x@50000000/dc@54240000...

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd071]
[    0.000000] Linux version 5.15.50 (koen@koen-nuc11) (aarch64-openwrt-linux-musl-gcc (OpenWrt GCC 11.3.0 r0+20012-0781fb22fa) 11.3.0, GNU ld (GNU Binutils) 2.37) #0 SMP Thu Jul 7 15:02:25 2022
[    0.000000] Machine model: NVIDIA Jetson Nano Developer Kit
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000070006000 (options '')
[    0.000000] printk: bootconsole [uart8250] enabled
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000afffffff]
[    0.000000]   node   0: [mem 0x00000000b0000000-0x00000000b01fffff]
[    0.000000]   node   0: [mem 0x00000000b0200000-0x00000000fedfffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000017f1fffff]
[    0.000000] On node 0, zone Normal: 4608 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 3584 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 17 pages/cpu s31000 r8192 d30440 u69632
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: Spectre-v2
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1024056
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=u0
[    0.000000] Unknown kernel command line parameters "vpr_resize tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 lp0_vec=0x1000.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000fae00000-0x00000000fee00000] (64MB)
[    0.000000] Memory: 3581744K/4161536K available (8832K kernel code, 1012K rwdata, 2556K rodata, 512K init, 311K bss, 579792K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: GICv2 detected, but range too small and irqchip.gicv2_force_probe not set
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000000] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.008311] tegra-timer: failed to map irq for cpu0
[    0.013348] Failed to initialize '/timer@60005000': -22
[    0.019859] Console: colour dummy device 80x25
[    0.024452] printk: console [tty0] enabled
[    0.028673] printk: bootconsole [uart8250] disabled
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd071]
[    0.000000] Linux version 5.15.50 (koen@koen-nuc11) (aarch64-openwrt-linux-musl-gcc (OpenWrt GCC 11.3.0 r0+20012-0781fb22fa) 11.3.0, GNU ld (GNU Binutils) 2.37) #0 SMP Thu Jul 7 15:02:25 2022
[    0.000000] Machine model: NVIDIA Jetson Nano Developer Kit
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000070006000 (options '')
[    0.000000] printk: bootconsole [uart8250] enabled
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000afffffff]
[    0.000000]   node   0: [mem 0x00000000b0000000-0x00000000b01fffff]
[    0.000000]   node   0: [mem 0x00000000b0200000-0x00000000fedfffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000017f1fffff]
[    0.000000] On node 0, zone Normal: 4608 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 3584 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 17 pages/cpu s31000 r8192 d30440 u69632
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: Spectre-v2
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1024056
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=u0
[    0.000000] Unknown kernel command line parameters "vpr_resize tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 lp0_vec=0x1000.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000fae00000-0x00000000fee00000] (64MB)
[    0.000000] Memory: 3581744K/4161536K available (8832K kernel code, 1012K rwdata, 2556K rodata, 512K init, 311K bss, 579792K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: GICv2 detected, but range too small and irqchip.gicv2_force_probe not set
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000000] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.008311] tegra-timer: failed to map irq for cpu0
[    0.013348] Failed to initialize '/timer@60005000': -22
[    0.019859] Console: colour dummy device 80x25
[    0.024452] printk: console [tty0] enabled
[    0.028673] printk: bootconsole [uart8250] disabled
[    0.033742] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.033763] pid_max: default: 32768 minimum: 301
[    0.033993] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.034060] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.035764] rcu: Hierarchical SRCU implementation.
[    0.035933] dyndbg: Ignore empty _ddebug table in a CONFIG_DYNAMIC_DEBUG_CORE build
[    0.039303] Speedo Revision 4
[    0.039318] Tegra210: unknown SKU 0x8f
[    0.039338] Tegra Revision: A02 SKU: 143 CPU Process: 0 SoC Process: 0
[    0.040272] tegra-pmc: access to PMC is restricted to TZ
[    0.040623] smp: Bringing up secondary CPUs ...
[    0.041196] Detected PIPT I-cache on CPU1
[    0.041284] CPU1: Booted secondary processor 0x0000000001 [0x411fd071]
[    0.041900] Detected PIPT I-cache on CPU2
[    0.041936] CPU2: Booted secondary processor 0x0000000002 [0x411fd071]
[    0.042513] Detected PIPT I-cache on CPU3
[    0.042547] CPU3: Booted secondary processor 0x0000000003 [0x411fd071]
[    0.042606] smp: Brought up 1 node, 4 CPUs
[    0.042653] SMP: Total of 4 processors activated.
[    0.042669] CPU features: detected: 32-bit EL0 Support
[    0.042679] CPU features: detected: CRC32 instructions
[    0.042735] CPU features: emulated: Privileged Access Never (PAN) using TTBR0_EL1 switching
[    0.042751] CPU: All CPU(s) started at EL2
[    0.042774] alternatives: patching kernel code
[    0.063925] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.063987] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.064198] pinctrl core: initialized pinctrl subsystem
[    0.065416] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.066061] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.066167] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.066251] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.066636] thermal_sys: Registered thermal governor 'step_wise'
[    0.068574] ASID allocator initialised with 65536 entries
[    0.079793] tegra210-pinctrl 700008d4.pinmux: there is not valid maps for state drive
[    0.097348] tegra-mc 70019000.memory-controller: no memory timings for RAM code 0 registered
[    0.112958] cryptd: max_cpu_qlen set to 1000
[    0.115030] vdd-5v0-hdmi: supplied by vdd-5v0-sys
[    0.115303] vdd-fan: supplied by vdd-5v0-sys
[    0.115495] vdd-usb-vbus: supplied by vdd-5v0-sys
[    0.115848] iommu: Default domain type: Translated 
[    0.115858] iommu: DMA domain TLB invalidation policy: strict mode 
[    0.116128] SCSI subsystem initialized
[    0.116482] usbcore: registered new interface driver usbfs
[    0.116524] usbcore: registered new interface driver hub
[    0.116556] usbcore: registered new device driver usb
[    0.116817] pps_core: LinuxPPS API ver. 1 registered
[    0.116830] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.116849] PTP clock support registered
[    0.117189] Advanced Linux Sound Architecture Driver Initialized.
[    0.118066] clocksource: Switched to clocksource arch_sys_counter
[    0.121009] NET: Registered PF_INET protocol family
[    0.121558] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.123354] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.123429] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.123449] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.123667] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.124236] TCP: Hash tables configured (established 32768 bind 32768)
[    0.124472] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.124573] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.124840] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.124890] PCI: CLS 0 bytes, default 64
[    0.125538] Unpacking initramfs...
[    0.406921] Freeing initrd memory: 6988K
[    0.409456] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.412658] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.412741] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    0.422457] tegra-apbdma 60020000.dma: Tegra20 APB DMA driver registered 32 channels
[    0.425921] tegra-pmc 7000e400.pmc: i2c-thermtrip node not found, emergency thermal reset disabled.
[    0.427413] Serial: 8250/16550 driver, 8 ports, IRQ sharing disabled
[    0.436510] loop: module loaded
[    0.440446] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.440493] ehci-platform: EHCI generic platform driver
[    0.440729] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.440759] ohci-platform: OHCI generic platform driver
[    0.441233] usbcore: registered new interface driver usb-storage
[    0.442053] mousedev: PS/2 mouse device common for all mice
[    0.442180] i2c_dev: i2c /dev entries driver
[    0.443691] tegra-wdt 60005000.timer: initialized (heartbeat = 120 sec, nowayout = 0)
[    0.445516] sdhci: Secure Digital Host Controller Interface driver
[    0.445545] sdhci: Copyright(c) Pierre Ossman
[    0.445553] sdhci-pltfm: SDHCI platform and OF driver helper
[    0.446586] tegra210-emc: probe of 7001b000.external-memory-controller failed with error -2
[    0.450447] NET: Registered PF_INET6 protocol family
[    0.451764] Segment Routing with IPv6
[    0.451813] In-situ OAM (IOAM) with IPv6
[    0.451873] NET: Registered PF_PACKET protocol family
[    0.451929] can: controller area network core
[    0.452036] NET: Registered PF_CAN protocol family
[    0.452050] 8021q: 802.1Q VLAN Support v1.8
[    0.469163] gpio-151 (camera-control-output-low): hogged as output/low
[    0.469224] gpio-152 (camera-control-output-low): hogged as output/low
[    0.469251] gpio-6 (system-suspend-gpio): hogged as output/high
[    0.472390] tegra-uart 70006000.serial: deferred probe timeout, ignoring dependency
[    0.472769] printk: console [ttyS0] disabled
[    0.472840] 70006000.serial: ttyS0 at MMIO 0x70006000 (irq = 67, base_baud = 25500000) is a Tegra
[    1.471766] printk: console [ttyS0] enabled
[    1.476418] tegra-qspi 70410000.spi: deferred probe timeout, ignoring dependency
[    1.485034] tegra-i2c 7000c000.i2c: deferred probe timeout, ignoring dependency
[    1.493061] tegra-i2c 7000c400.i2c: deferred probe timeout, ignoring dependency
[    1.500958] tegra-i2c 7000c500.i2c: deferred probe timeout, ignoring dependency
[    1.508814] tegra-i2c 7000c700.i2c: deferred probe timeout, ignoring dependency
[    1.516630] tegra-i2c 7000d000.i2c: deferred probe timeout, ignoring dependency
[    1.525655] tegra-i2c 7000d100.i2c: deferred probe timeout, ignoring dependency
[    1.537380] ALSA device list:
[    1.540432]   No soundcards found.
[    1.544466] Freeing unused kernel memory: 512K
[    1.618180] Run /init as init process
[   31.848093] vdd-5v0-sys: disabling
[   31.851528] vdd-5v0-hdmi: disabling
[   31.855014] vdd-fan: disabling
[   31.858081] vdd-usb-vbus: disabling

xback · July 12, 2022, 8:48am

2nd test. Fully booting OpenWRT

U-Boot 2020.04-g6b630d64fd (Feb 19 2021 - 08:37:46 -0800)

SoC: tegra210
Model: NVIDIA Jetson Nano Developer Kit
Board: NVIDIA P3450-0000
DRAM:  4 GiB
MMC:   sdhci@700b0000: 1, sdhci@700b0600: 0
Loading Environment from SPI Flash... SF: Detected mx25u3235f with page size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot.scr
375 bytes read in 9 ms (40 KiB/s)
## Executing script at 90000000
13572104 bytes read in 597 ms (21.7 MiB/s)
46822 bytes read in 14 ms (3.2 MiB/s)
## Flattened Device Tree blob at 83000000
   Booting using the fdt blob at 0x83000000
   Using Device Tree in place at 0000000083000000, end 000000008300e6e5
copying carveout for /host1x@50000000/dc@54200000...
copying carveout for /host1x@50000000/dc@54240000...
Can't set DT prop /bpmp/carveout-start: FDT_ERR_BADOFFSET

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd071]
[    0.000000] Linux version 5.15.53 (koen@koen-nuc11) (aarch64-openwrt-linux-musl-gcc (OpenWrt GCC 12.1.0 r20010+61-6d423ffbd1) 12.1.0, GNU ld (GNU Binutils) 2.38) #0 SMP Tue Jul 12 09:15:54 2022
[    0.000000] Machine model: NVIDIA Jetson Nano Developer Kit
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000fedfffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000017f1fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000017f1fffff]
[    0.000000] On node 0, zone Normal: 4608 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 3584 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 17 pages/cpu s31000 r8192 d30440 u69632
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: Spectre-v2
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1024056
[    0.000000] Kernel command line: root=PARTUUID=1c9673b0-02 rw rootwait console=ttyS0,115200 console=tty0
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000fae00000-0x00000000fee00000] (64MB)
[    0.000000] Memory: 4001072K/4161536K available (8896K kernel code, 1030K rwdata, 2728K rodata, 512K init, 309K bss, 160464K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller@50041000
[    0.000002] sched_clock: 32 bits at 1000kHz, resolution 1000ns, wraps every 2147483647500ns
[    0.000030] clocksource: timer_us: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
[    0.000184] clocksource: tegra_suspend_timer: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.000711] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000723] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000735] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.000867] Console: colour dummy device 80x25
[    0.001099] printk: console [tty0] enabled
[    0.001151] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.001172] pid_max: default: 32768 minimum: 301
[    0.001395] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001464] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.003208] rcu: Hierarchical SRCU implementation.
[    0.003389] dyndbg: Ignore empty _ddebug table in a CONFIG_DYNAMIC_DEBUG_CORE build
[    0.004296] Speedo Revision 4
[    0.004311] Tegra210: unknown SKU 0x8f
[    0.004332] Tegra Revision: A02 SKU: 143 CPU Process: 0 SoC Process: 0
[    0.004603] tegra-pmc: access to PMC is restricted to TZ
[    0.004907] smp: Bringing up secondary CPUs ...
[    0.005474] Detected PIPT I-cache on CPU1
[    0.005566] CPU1: Booted secondary processor 0x0000000001 [0x411fd071]
[    0.006180] Detected PIPT I-cache on CPU2
[    0.006221] CPU2: Booted secondary processor 0x0000000002 [0x411fd071]
[    0.006813] Detected PIPT I-cache on CPU3
[    0.006857] CPU3: Booted secondary processor 0x0000000003 [0x411fd071]
[    0.006921] smp: Brought up 1 node, 4 CPUs
[    0.006970] SMP: Total of 4 processors activated.
[    0.006980] CPU features: detected: 32-bit EL0 Support
[    0.006990] CPU features: detected: CRC32 instructions
[    0.007052] CPU features: emulated: Privileged Access Never (PAN) using TTBR0_EL1 switching
[    0.007069] CPU: All CPU(s) started at EL2
[    0.007092] alternatives: patching kernel code
[    0.015435] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.015494] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.015698] pinctrl core: initialized pinctrl subsystem
[    0.016964] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.017486] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.017583] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.017661] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.018044] thermal_sys: Registered thermal governor 'step_wise'
[    0.018613] OF: /thermal-zones/mem/cooling-maps/dram-passive: could not find phandle 133
[    0.018659] thermal_sys: Add a cooling_device property with at least one device
[    0.018669] thermal_sys: failed to build thermal zone mem: -2
[    0.018908] ASID allocator initialised with 65536 entries
[    0.027420] tegra-mc 70019000.memory-controller: no memory timings for RAM code 0 registered
[    0.027952] platform 50000000.host1x: Adding to iommu group 0
[    0.028003] platform 57000000.gpu: Adding to iommu group 1
[    0.028279] OF: /external-memory-controller@7001b000: could not get #clock-cells for /i2c@7000d000/pmic@3c/pinmux
[    0.041159] cryptd: max_cpu_qlen set to 1000
[    0.042770] VDD_HDMI_5V0: supplied by VDD_5V0_SYS
[    0.043048] VDD_5V_USB: supplied by VDD_5V0_SYS
[    0.043223] iommu: Default domain type: Translated 
[    0.043235] iommu: DMA domain TLB invalidation policy: strict mode 
[    0.043652] SCSI subsystem initialized
[    0.043952] usbcore: registered new interface driver usbfs
[    0.043990] usbcore: registered new interface driver hub
[    0.044023] usbcore: registered new device driver usb
[    0.044196] pps_core: LinuxPPS API ver. 1 registered
[    0.044207] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.044226] PTP clock support registered
[    0.044506] Advanced Linux Sound Architecture Driver Initialized.
[    0.045233] clocksource: Switched to clocksource arch_sys_counter
[    0.048042] NET: Registered PF_INET protocol family
[    0.048556] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.050387] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.050464] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.050486] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.050708] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.051229] TCP: Hash tables configured (established 32768 bind 32768)
[    0.051443] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.051544] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.051817] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.051870] PCI: CLS 0 bytes, default 64
[    0.056173] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.059302] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.059391] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    0.067023] tegra-apbdma 60020000.dma: Tegra20 APB DMA driver registered 32 channels
[    0.070786] tegra-pmc 7000e400.pmc: i2c-thermtrip node not found, emergency thermal reset disabled.
[    0.072215] Serial: 8250/16550 driver, 8 ports, IRQ sharing disabled
[    0.073694] printk: console [ttyS0] disabled
[    0.073781] 70006000.serial: ttyS0 at MMIO 0x70006000 (irq = 74, base_baud = 25500000) is a Tegra
[    0.837760] printk: console [ttyS0] enabled
[    0.843537] Failed to set up IOMMU for device 50000000.host1x; retaining platform DMA ops
[    0.853627] platform 54200000.dc: Adding to iommu group 2
[    0.859329] platform 54240000.dc: Adding to iommu group 2
[    0.864959] platform 54340000.vic: Adding to iommu group 3
[    0.876661] loop: module loaded
[    0.882395] spi-nor spi0.0: mx25u3235f (4096 Kbytes)
[    0.890325] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.896907] ehci-platform: EHCI generic platform driver
[    0.902277] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.908474] ohci-platform: OHCI generic platform driver
[    0.914010] usbcore: registered new interface driver usb-storage
[    0.920575] mousedev: PS/2 mouse device common for all mice
[    0.926221] i2c_dev: i2c /dev entries driver
[    0.934336] max77620 1-003c: PMIC Version OTP:0x35 and ES:0x8
[    0.946960] VDD_SOC: supplied by VDD_5V0_SYS
[    0.952621] VDD_DDR_1V1_PMIC: Bringing 1125000uV into 1150000-1150000uV
[    0.959982] VDD_DDR_1V1_PMIC: supplied by VDD_5V0_SYS
[    0.966405] VDD_PRE_REG_1V35: supplied by VDD_5V0_SYS
[    0.972836] VDD_1V8: supplied by VDD_5V0_SYS
[    0.978565] AVDD_SYS_1V2: supplied by VDD_PRE_REG_1V35
[    0.984735] VDD_PEX_1V05: supplied by VDD_PRE_REG_1V35
[    0.991003] max77620-ldo3: supplied by VDD_1V8
[    0.996171] VDD_RTC: supplied by VDD_5V0_SYS
[    1.001199] max77620-ldo5: supplied by VDD_1V8
[    1.006137] max77620-ldo6: supplied by VDD_5V0_SYS
[    1.011753] AVDD_1V05_PLL: supplied by VDD_PRE_REG_1V35
[    1.017837] AVDD_SATA_HDMI_DP_1V05: supplied by VDD_PRE_REG_1V35
[    1.028784] max77620-thermal max77620-thermal: Failed to register thermal zone: -19
[    1.036724] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    1.047263] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    1.056751] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    1.066126] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    1.075605] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    1.085054] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    1.094585] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    1.104513] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 1017600 KHz, changing to: 1020000 KHz
[    1.115757] sdhci: Secure Digital Host Controller Interface driver
[    1.121939] sdhci: Copyright(c) Pierre Ossman
[    1.126431] Synopsys Designware Multimedia Card Interface Driver
[    1.132502] VUB300 Driver rom wait states = 1C irqpoll timeout = 0400
[    1.132754] usbcore: registered new interface driver vub300
[    1.144775] usbcore: registered new interface driver ushc
[    1.150171] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.156288] OF: /external-memory-controller@7001b000: could not get #clock-cells for /i2c@7000d000/pmic@3c/pinmux
[    1.166622] tegra210-emc: probe of 7001b000.external-memory-controller failed with error -2
[    1.176541] NET: Registered PF_INET6 protocol family
[    1.182239] Segment Routing with IPv6
[    1.185949] In-situ OAM (IOAM) with IPv6
[    1.189912] NET: Registered PF_PACKET protocol family
[    1.194986] can: controller area network core
[    1.199431] NET: Registered PF_CAN protocol family
[    1.204212] 8021q: 802.1Q VLAN Support v1.8
[    1.221198] VDD_3V3_SYS: supplied by VDD_5V0_SYS
[    1.227426] VDD_CPU: supplied by VDD_5V0_SYS
[    1.231994] VDDIO_SDMMC: supplied by VDD_3V3_SYS
[    1.237403] AVDD_IO_EDP_1V05: supplied by AVDD_1V05_PLL
[    1.243509] gic 702f9000.interrupt-controller: GIC IRQ controller registered
[    1.251159] tegra-aconnect aconnect@702c0000: Tegra ACONNECT bus registered
[    1.259892]  usb2-1: supply vbus not found, using dummy regulator
[    1.266153]  usb2-2: supply vbus not found, using dummy regulator
[    1.274939] tegra-pcie 1003000.pcie: host bridge /pcie@1003000 ranges:
[    1.281519] tegra-pcie 1003000.pcie:      MEM 0x0001000000..0x0001001fff -> 0x0001000000
[    1.289607] tegra-pcie 1003000.pcie:       IO 0x0012000000..0x001200ffff -> 0x0000000000
[    1.297695] tegra-pcie 1003000.pcie:      MEM 0x0013000000..0x001fffffff -> 0x0013000000
[    1.305776] tegra-pcie 1003000.pcie:      MEM 0x0020000000..0x003fffffff -> 0x0020000000
[    1.313921] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[    1.319772] tegra-pcie 1003000.pcie: failed to get PHY#0: -517
[    1.325606] tegra-pcie 1003000.pcie: failed to get PHYs: -517
[    1.331338] tegra-pcie 1003000.pcie: failed to request resources: -517
[    1.339131] at24 0-0050: 256 byte 24c02 EEPROM, read-only
[    1.345250] at24 0-0057: 256 byte 24c02 EEPROM, read-only
[    1.350960] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    1.361675] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    1.371155] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    1.380509] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    1.389978] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    1.399397] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    1.408894] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    1.417126] VDD_3V3_SD: supplied by VDD_3V3_SYS
[    1.421897] VDD_HUB_3V3: supplied by VDD_5V0_SYS
[    1.428562] tegra-adma 702e2000.dma-controller: Tegra210 ADMA driver registered 22 channels
[    1.439726] irq: IRQ133: trimming hierarchy from :pmc@7000e400
[    1.445977]  usb2-1: supply vbus not found, using dummy regulator
[    1.452180]  usb2-2: supply vbus not found, using dummy regulator
[    1.458982] tegra-pcie 1003000.pcie: host bridge /pcie@1003000 ranges:
[    1.465544] tegra-pcie 1003000.pcie:      MEM 0x0001000000..0x0001001fff -> 0x0001000000
[    1.473624] tegra-pcie 1003000.pcie:       IO 0x0012000000..0x001200ffff -> 0x0000000000
[    1.481715] tegra-pcie 1003000.pcie:      MEM 0x0013000000..0x001fffffff -> 0x0013000000
[    1.489802] tegra-pcie 1003000.pcie:      MEM 0x0020000000..0x003fffffff -> 0x0020000000
[    1.497959] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[    1.507029] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
[    1.515251] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
[    1.529983] mmc0: SDHCI controller on 700b0400.mmc [700b0400.mmc] using ADMA 64-bit
[    2.732963] tegra-pcie 1003000.pcie: link 0 down, ignoring
[    2.748617] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
[    2.754872] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.760356] pci_bus 0000:00: root bus resource [mem 0x01000000-0x01001fff]
[    2.767221] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    2.773389] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[    2.780251] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[    2.788195] pci 0000:00:02.0: [10de:0faf] type 01 class 0x060400
[    2.794200] pci_bus 0000:00: 2-byte config write to 0000:00:02.0 offset 0x4 may corrupt adjacent RW1C bits
[    2.803874] pci 0000:00:02.0: enabling Extended Tags
[    2.808885] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[    2.817090] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    2.825228] pci 0000:01:00.0: [10ec:8168] type 00 class 0x020000
[    2.831247] pci 0000:01:00.0: reg 0x10: [io  0x0000-0x00ff]
[    2.836842] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[    2.843635] pci 0000:01:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[    2.850552] pci 0000:01:00.0: supports D1 D2
[    2.854812] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    2.862916] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    2.869552] pci 0000:00:02.0: BAR 8: assigned [mem 0x13000000-0x130fffff]
[    2.876331] pci 0000:00:02.0: BAR 7: assigned [io  0x1000-0x1fff]
[    2.882416] pci 0000:01:00.0: BAR 4: assigned [mem 0x13000000-0x13003fff 64bit]
[    2.889731] pci 0000:01:00.0: BAR 2: assigned [mem 0x13004000-0x13004fff 64bit]
[    2.897042] pci 0000:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    2.903127] pci 0000:00:02.0: PCI bridge to [bus 01]
[    2.908084] pci 0000:00:02.0:   bridge window [io  0x1000-0x1fff]
[    2.914166] pci 0000:00:02.0:   bridge window [mem 0x13000000-0x130fffff]
[    2.920966] pci 0000:00:02.0: nv_msi_ht_cap_quirk didn't locate host bridge
[    2.933636] tegra-xusb 70090000.usb: Firmware timestamp: 2015-08-10 09:47:54 UTC
[    2.941068] tegra-xusb 70090000.usb: xHCI Host Controller
[    2.946484] tegra-xusb 70090000.usb: new USB bus registered, assigned bus number 1
[    2.954885] tegra-xusb 70090000.usb: hcc params 0x0184f525 hci version 0x100 quirks 0x0000000000010010
[    2.964235] tegra-xusb 70090000.usb: irq 81, io mem 0x70090000
[    2.970367] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15
[    2.978636] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.985849] usb usb1: Product: xHCI Host Controller
[    2.990716] usb usb1: Manufacturer: Linux 5.15.53 xhci-hcd
[    2.996191] usb usb1: SerialNumber: 70090000.usb
[    3.001092] hub 1-0:1.0: USB hub found
[    3.004866] hub 1-0:1.0: 5 ports detected
[    3.009216] tegra-xusb 70090000.usb: xHCI Host Controller
[    3.014615] tegra-xusb 70090000.usb: new USB bus registered, assigned bus number 2
[    3.022185] tegra-xusb 70090000.usb: Host supports USB 3.0 SuperSpeed
[    3.028679] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    3.036823] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.15
[    3.045074] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.052287] usb usb2: Product: xHCI Host Controller
[    3.057156] usb usb2: Manufacturer: Linux 5.15.53 xhci-hcd
[    3.062627] usb usb2: SerialNumber: 70090000.usb
[    3.067381] hub 2-0:1.0: USB hub found
[    3.071143] hub 2-0:1.0: 4 ports detected
[    3.077517] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.088278] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.097759] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.107104] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.116579] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.125997] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.135496] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.143712] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.143986] sdhci-tegra 700b0000.mmc: Got CD GPIO
[    3.154212] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.168162] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.177579] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.187052] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.196485] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.205984] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.209975] irq: IRQ134: trimming hierarchy from :pmc@7000e400
[    3.214965] ALSA device list:
[    3.219685] mmc1: SDHCI controller on 700b0000.mmc [700b0000.mmc] using ADMA 64-bit
[    3.222445]   No soundcards found.
[    3.230551] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.244092] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.253575] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.262914] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.272382] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.281798] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.291296] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.299125] Waiting for root device PARTUUID=1c9673b0-02...
[    3.305248] usb 1-2: new high-speed USB device number 2 using tegra-xusb
[    3.493452] usb 1-2: New USB device found, idVendor=0bda, idProduct=5411, bcdDevice= 1.01
[    3.501633] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    3.508758] usb 1-2: Product: 4-Port USB 2.1 Hub
[    3.513365] usb 1-2: Manufacturer: Generic
[    3.519236] hub 1-2:1.0: USB hub found
[    3.523963] hub 1-2:1.0: 4 ports detected
[    3.531105] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.541674] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.551171] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.560518] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.569993] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.579417] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.588923] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.607961] mmc1: new ultra high speed SDR104 SDHC card at address aaaa
[    3.615241] mmcblk1: mmc1:aaaa SC32G 29.7 GiB 
[    3.625044]  mmcblk1: p1 p2
[    3.628525] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.639151] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.648631] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.657970] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.667438] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.676853] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.686350] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.694641] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[    3.732606] VFS: Mounted root (squashfs filesystem) readonly on device 179:2.
[    3.734121] usb 2-1: New USB device found, idVendor=0bda, idProduct=0411, bcdDevice= 1.01
[    3.740188] Freeing unused kernel memory: 512K
[    3.747913] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    3.759470] usb 2-1: Product: 4-Port USB 3.1 Hub
[    3.764076] usb 2-1: Manufacturer: Generic
[    3.772421] hub 2-1:1.0: USB hub found
[    3.776966] hub 2-1:1.0: 4 ports detected
[    3.783549] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    3.794200] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    3.803681] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    3.813017] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    3.822485] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    3.831899] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    3.841397] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    3.855325] Run /sbin/init as init process
[    3.977892] init: Console is alive
[    3.981820] init: - watchdog -
[    4.183458] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    4.216979] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    4.234727] init: - preinit -
[    4.406326] random: jshn: uninitialized urandom read (4 bytes read)
[    4.417639] random: jshn: uninitialized urandom read (4 bytes read)
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
[    6.474334] loop0: detected capacity change from 0 to 212992
[    6.489680] random: procd: uninitialized urandom read (4 bytes read)
[    6.555243] loop0: detected capacity change from 212992 to 203648
[    6.590892] EXT4-fs (loop0): recovery complete
[    6.596766] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[    6.607013] mount_root: switching to ext4 overlay
[    6.615930] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
[    6.675455] EXT4-fs (mmcblk1p1): mounted filesystem without journal. Opts: (null). Quota mode: disabled.
[    6.692273] urandom-seed: Seeding with /etc/urandom.seed
[    6.740286] procd: - early -
[    6.743269] procd: - watchdog -
[    7.281067] procd: - watchdog -
[    7.311774] procd: - ubus -
[    7.338291] random: ubusd: uninitialized urandom read (4 bytes read)
[    7.368470] random: ubusd: uninitialized urandom read (4 bytes read)
[    7.380377] procd: - init -
Please press Enter to activate this console.
[    7.495294] random: ubusd: uninitialized urandom read (4 bytes read)
[    7.501858] random: ubus: uninitialized urandom read (4 bytes read)
[    7.536107] random: jshn: uninitialized urandom read (4 bytes read)
[    7.549766] urngd: v1.0.2 started.
[    7.558244] kmodloader: loading kernel modules from /etc/modules.d/*
[    7.567036] random: crng init done
[    7.570452] random: 20 urandom warning(s) missed due to ratelimiting
[    7.602322] Loading modules backported from Linux version v5.15.33-0-g06f50ca83ace
[    7.609963] Backport generated by backports.git v5.15.33-1-0-g183c4ab2
[    7.622176] pci 0000:00:02.0: enabling device (0000 -> 0003)
[    7.627960] r8169 0000:01:00.0: enabling device (0000 -> 0003)
[    7.681259] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    7.681259] r8169 0000:01:00.0 eth0: RTL8168h/8111h, 00:04:4b:ec:31:2c, XID 541, IRQ 135
[    7.682049] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    7.691592] r8169 0000:01:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[    7.699708] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    7.727508] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    7.737038] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    7.746814] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    7.756587] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    7.765133] tegra_soctherm 700e2000.thermal-sensor: missing thermtrips, will use critical trips as shut down temp
[    7.776303] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when cpu reaches 102500 mC
[    7.785889] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when cpu reaches 98500 mC
[    7.795489] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when gpu reaches 103000 mC
[    7.804980] tegra_soctherm 700e2000.thermal-sensor: throttrip: will throttle when gpu reaches 100000 mC
[    7.814639] tegra_soctherm 700e2000.thermal-sensor: thermtrip: will shut down when pll reaches 103000 mC
[    7.824244] tegra_soctherm 700e2000.thermal-sensor: failed to register sensor: -517
[    7.838763] PPP generic driver version 2.4.2
[    7.843671] NET: Registered PF_PPPOX protocol family
[    7.855128] usbcore: registered new interface driver rt2800usb
[    7.861181] kmodloader: done loading kernel modules from /etc/modules.d/*
[   10.525538] Generic FE-GE Realtek PHY r8169-0-100:00: attached PHY driver (mii_bus:phy_addr=r8169-0-100:00, irq=MAC)
[   10.805346] r8169 0000:01:00.0 eth0: Link is Down
[   10.810076] br-lan: port 1(eth0) entered blocking state
[   10.815333] br-lan: port 1(eth0) entered disabled state
[   10.820794] device eth0 entered promiscuous mode
[   10.826656] br-lan: port 1(eth0) entered blocking state
[   10.831881] br-lan: port 1(eth0) entered forwarding state
[   11.515401] br-lan: port 1(eth0) entered disabled state
[   14.481305] r8169 0000:01:00.0 eth0: Link is Up - 1Gbps/Full - flow control off
[   14.488726] br-lan: port 1(eth0) entered blocking state
[   14.493947] br-lan: port 1(eth0) entered forwarding state
[   14.499842] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready



BusyBox v1.35.0 (2022-07-12 09:15:54 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt SNAPSHOT, r20010+90-6d423ffbd1
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# lscpu
Architecture:            aarch64
  CPU op-mode(s):        32-bit, 64-bit
  Byte Order:            Little Endian                                                                                                                
CPU(s):                  4                                                                                                                            
  On-line CPU(s) list:   0-3                                                                                                                          
Vendor ID:               ARM                                                                                                                          
  Model name:            Cortex-A57
    Model:               1
    Thread(s) per core:  1
    Core(s) per cluster: 4
    Socket(s):           -
    Cluster(s):          1
    Stepping:            r1p1
    CPU(s) scaling MHz:  100%
    CPU max MHz:         1912.5000
    CPU min MHz:         204.0000
    BogoMIPS:            38.40
    Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
Vulnerabilities:         
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Spec store bypass:     Not affected
  Spectre v1:            Mitigation; __user pointer sanitization
  Spectre v2:            Mitigation; Branch predictor hardening, BHB
  Srbds:                 Not affected
  Tsx async abort:       Not affected
root@OpenWrt:/# lspci -v
00:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1) (prog-if 00 [Normal decode])
        Device tree node: /sys/firmware/devicetree/base/pcie@1003000/pci@2,0
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00001fff [size=4K]
        Memory behind bridge: 13000000-130fffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [40] Subsystem: NVIDIA Corporation Device 0000
        Capabilities: [48] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+
        Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
        Capabilities: [80] Express Root Port (Slot+), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] L1 PM Substates
lspci: Unable to load libkmod resources: error -12

01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
        Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller
        Device tree node: /sys/firmware/devicetree/base/pcie@1003000/pci@2,0/ethernet@0,0
        Flags: bus master, fast devsel, latency 0, IRQ 18
        I/O ports at 1000 [size=256]
        Memory at 13004000 (64-bit, non-prefetchable) [size=4K]
        Memory at 13000000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 01
        Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
        Capabilities: [d0] Vital Product Data
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [160] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [170] Latency Tolerance Reporting
        Capabilities: [178] L1 PM Substates
        Kernel driver in use: r8169

root@OpenWrt:/# lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=, Driver=hub/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/5p, 480M
    |__ Port 2: Dev 2, If 0, Class=, Driver=hub/4p, 480M
root@OpenWrt:/#

xback · July 16, 2022, 8:03pm

Hi,

Just as a follow-up:

Base:

Backported all changes from >5.15 kernels to fix dts(i) stuff
Fixed some issues which show up in above bootlog, but still some remaining.
The fan also works
Optimized the kernel config more by reducing the amount of unrequired drivers (like 100g ethernet stuff etc), reducing the image size and compile time
Added all latest tegra firmware files to the rootfs
Lots of cleanups

GPU:
I’ve actually ported both public source versions (nvgpu) to kernel 5.15 so ending up with 2 patchsets:

34.1.1 is done and compiles, fixing 3 bugs in the 34.1.1 public sources.
32.7.1 is 90% done. Only some specific DMA code remaining
Comparing both in detail shows that 34.1.1 is a huge improvement in both code quality and optimization

Next steps:

Fix remainingbootlog issues seen
Implement OpenWRT platform upgrade for this target
Fix some neon asm issues when building gstreamer 1.20.x on aarch64
…

xback · July 24, 2022, 8:31pm

Just a minor update to avoid thread closure.

I’m currently on holiday and will continue working on this subject next week. (Aug 1st)

nguoido97 · August 6, 2022, 4:14pm

Hi xback,

I would like to join to support nano on openwrt with you

xback · August 9, 2022, 1:39pm

Great!

Work on including GPU code etc is still ongoing.
I also requested for the dts file for the 2G flavor of the devkit.

Regards

paralin · September 14, 2022, 4:12am

Thanks for your work on this, it would be great to use a newer kernel than the 4.9.x, and porting it to 5.10 is not a trivial task.

xback · September 20, 2022, 6:59pm

Apologies for the late reply,
and thanks for not closing yet.

I needed to work on something else first.

Will continue on this topic now…

In the meantime, I investigated the carveout mechanism in use to better understand the full boot sequence. (And thus how the gpu binary is delivered to the engine)

I also prepped a few TX2’s on CTI spacely carrier boards to test everything on 3rd party hardware too. (As I have a lot of those laying around)

No word yet from upstream regarding an open dts for nano 2gb.
Will shake that tree again hoping something drops out.

Regards

xback · September 20, 2022, 7:02pm

I’m porting everything to 5.15 actually as that LTS is now in use on OpenWRT master.

I noticed the Tegra target got bumped to 5.15 so I’ll move the current code from a separate target to that one.

xback · February 13, 2023, 8:11pm

Hi all,

To provide some further updates.
I’ve been out for a few months due to medical reasons.

fully working with kernel 5.15
fully working gstreamer1.20.5
host1x interface etc fully functional.
working on properly flashing to make it compatible with the carveouts. (No cboot sources available for nano? Why?)
working on a 2nd device based on tx2 and cti spacely carrier as I have lots of those for stress testing.

Some issues seen:

backports are needed currently for dts fixes. It seems that the upstream nvidia people not always backport to previous LTS kernels.
Openwrt should support kernel 6.1 within a few weeks so the backports should get reduced automatically by then.
gpu support still ongoing. Firmware binaries are available in the build now.

Currently working on adding gstreamer 1.22.0 support.

xback · February 13, 2023, 8:15pm

Any update regarding the internal discussion?

Thanks!

DaneLLL · February 14, 2023, 2:41am

Hi,
Currently, the packages such as CUDA, TensorRT are in fixed versions in Jetpack releases. For a long term plan, we are evaluating to have more flexibility in the package versions.

xback · February 14, 2023, 6:18am

Thanks for the reply.

At first, I would try to add nvenc/nvdec support.
These seem to be the least invasive judging on code amount/complexity.

DaneLLL · February 14, 2023, 7:44am

Hi,

This is supported. We have hardware codec engines in Jetson chips so hardware encoding and decoding are supported. Supported software frameworks are jetson_multimedia_api and gstreamer.

xback · February 14, 2023, 8:28am

Hi @DaneLLL

Just to be sure we are on the same track:

Is it natively supported in upstream linux? (>=5.15) (see below dts)
Is it natively supported in upstream gstreamer? >=1.20

I know it works well in Jetpack, but the point is to have it working in OpenWRT without any jetpack involved :)
relying natively on kernel support exposing the required devs.

So basically:

nvenc/nvdec is accessed through host1x, right?
Some firmware needs to be uploaded to the IP block
Drivers need to be ported upstream? (is there any intention to do that from nvidia? even if only for jetson?)

Upstream DTS for t210:

nvdec@54480000 {
			compatible = "nvidia,tegra210-nvdec";
			reg = <0x0 0x54480000 0x0 0x00040000>;
			status = "disabled";
		};

		nvenc@544c0000 {
			compatible = "nvidia,tegra210-nvenc";
			reg = <0x0 0x544c0000 0x0 0x00040000>;
			status = "disabled";
		};

Thanks

DaneLLL · February 14, 2023, 9:16am

Hi,
Thanks for the explanation. We will evaluate this in future Jetpack release. You may consider use later Jetson platforms once this is enabled.