On a stock TX-1 with R24.2.1 and the standard development board it takes about 20 seconds to go from pressing the power button until the first messages come up on the HDMI display. After that it takes another 10 seconds to get to the Linux prompt (lightdm is turned off on this system). The total time to become operational is around 30 seconds. Our design goal is < 10 seconds. Can anyone comment on what is happening during the “dark” period? Is there a simple boot or build parameter that will eliminate what appears to be some sort of wait timeout?
With the serial console attached the boot process is captured below up to the start of the kernel. The elapsed time from pressing "Reset to Kernel Start is about 10 seconds with relatively long waits for two inputs and PCI identification.
Questions:
-
Is it SAFE to eliminate the PCI probe? We will not be using any PCI devices outside of the SOM. From the trace it does not appear any on-SOM PCI devices are discovered.
-
How do we get rid of the timeouts that correspond to the “Hit any key to sop autoboot” and “Enter Choice” prompts?
Thanks,
Spero
[0000.192] [TegraBoot] (version 24.00.2015.42-mobile-ec3b827e)
[0000.198] Processing in cold boot mode Bootloader 2
[0000.202] A02 Bootrom Patch rev = 63
[0000.206] Power-up reason: reset button
[0000.209] No Battery Present
[0000.212] RamCode = 0
[0000.214] Platform has Ddr4 type ram
[0000.217] max77620 disabling SD1 Remote Sense
[0000.222] Setting Ddr voltage to 1125mv
[0000.226] Serial Number of Pmic Max77663: 0x104f2
[0000.233] Entering ramdump check
[0000.236] Get RamDumpCarveOut = 0x0
[0000.239] RamDumpCarveOut=0x0, RamDumperFlag=0xe59ff3f8
[0000.245] Last reboot was clean, booting normally!
[0000.249] Sdram initialization is successful
[0000.253] SecureOs Carveout Base=0xff800000 Size=0x00800000
[0000.259] GSC1 Carveout Base=0xff700000 Size=0x00100000
[0000.264] GSC2 Carveout Base=0xff600000 Size=0x00100000
[0000.269] GSC3 Carveout Base=0xff500000 Size=0x00100000
[0000.274] GSC4 Carveout Base=0xff400000 Size=0x00100000
[0000.279] GSC5 Carveout Base=0xff300000 Size=0x00100000
[0000.284] BpmpFw Carveout Base=0xff2c0000 Size=0x00040000
[0000.290] Lp0 Carveout Base=0xff2bf000 Size=0x00001000
[0000.305] RamDump Carveout Base=0xff23f000 Size=0x00080000
[0000.310] Platform-DebugCarveout: 0
[0000.314] Nck Carveout Base=0xff03f000 Size=0x00200000
[0000.319] Non secure mode. Disable rollback prevention
[0000.324] AOTAG Init Done
[0000.371] Using GPT Primary to query partitions
[0000.377] Loading Tboot-CPU binary
[0000.426] Verifying bootloader in OdmNonSecureSBK mode
[0000.436] Bootloader load address is 0xa0000000, entry address is 0xa0000258
[0000.443] Bootloader downloaded successfully.
[0000.447] Downloaded Tboot-CPU binary to 0xa0000258
[0000.452] MAX77620_GPIO1 Configured.
[0000.455] MAX77620_GPIO5 Configured.
[0000.459] CPU power rail is up
[0000.462] CPU clock enabled
[0000.465] Performing RAM repair
[0000.468] Updating A64 Warmreset Address to 0xa00002e9
[0000.485] Bootloader DTB Load Address: 0x83000000
[0000.502] Kernel DTB Load Address: 0x83080000
[0000.507] Loading cboot binary
[0000.601] Verifying bootloader in OdmNonSecureSBK mode
[0000.695] Bootloader load address is 0x8010fda8, entry address is 0x80110000
[0000.702] Bootloader downloaded successfully.
[0000.707] GPT: Partition NOT found !
[0000.710] Find Partition via GPT Failed
[0000.714] Find Partition via PT Failed
[0000.717] function NvTbootGetBinaryOffsets: 0x1 error
[0000.722] Error in NvTbootLoadBinary: 0x1 !
[0000.726] Next binary entry address: 0x80110000
[0000.730] BoardId: 2180
[0000.756] NvTbootI2cProbe(): error code 0x00045100 Error while read
[0000.762] Display board id is not available
[0000.767] dram memory type is 3
[0000.771] WB0 init successful
[0000.797] Bpmp FW successfully loaded
[0000.800] Set NvDecSticky Bits
[0000.804] GSC1 address : ff700000
[0000.807] GSC2 address ff63fffc value c0edbbcc
[0000.812] GSC2 address : ff600000
[0000.815] GSC3 address : ff500000
[0000.819] GSC4 address : ff400000
[0000.823] GSC5 address : ff300000
[0000.826] GSC MC Settings done
[0000.830] TOS old plaintext Image length 65536
[0000.836] *** Secure OS image signature not verified ***
[0000.841] Loading and Validation of Secure OS Successful
[0000.846] NvTbootPackSdramParams: start.
[0000.851] NvTbootPackSdramParams: done.
[0000.855] Tegraboot started after 172287 us
[0000.859] Basic modules init took 313055 us
[0000.863] Sec Bootdevice Read Time = 194 ms, Read Size = 8464 KB
[0000.869] Sec Bootdevice Write Time = -1940251267 ms, Write Size = 343597383 KB
[0000.876] Next stage binary read took 12268 us
[0000.880] Carveout took 253097 us
[0000.883] CPU initialization took 125333 us
[0000.887] Total time taken by TegraBoot 703753 us
[0000.892] Starting CPU & Halting co-processor
64b[0001.000] RamCode = 0
[0001.015] LPDDR4 Training: Read DT: Number of tables = 10
[0001.020] EMC Training (SRC-freq: 204000; DST-freq: 40800)
[0001.025] EMC Training Skipped
[0001.028] EMC Training (SRC-freq: 204000; DST-freq: 68000)
[0001.033] EMC Training Skipped
[0001.036] EMC Training (SRC-freq: 204000; DST-freq: 102000)
[0001.041] EMC Training Skipped
[0001.044] EMC Training (SRC-freq: 204000; DST-freq: 204000)
[0001.049] EMC Training Skipped
[0001.052] EMC Training (SRC-freq: 204000; DST-freq: 408000)
[0001.058] EMC Training Successful
[0001.061] EMC Training (SRC-freq: 204000; DST-freq: 665600)
[0001.067] EMC Training Successful
[0001.070] EMC Training (SRC-freq: 204000; DST-freq: 800000)
[0001.081] EMC Training Successful
[0001.084] EMC Training (SRC-freq: 204000; DST-freq: 1065600)
[0001.107] EMC Training Successful
[0001.110] EMC Training (SRC-freq: 204000; DST-freq: 1331200)
[0001.132] EMC Training Successful
[0001.135] EMC Training (SRC-freq: 204000; DST-freq: 1600000)
[0001.154] EMC Training Successful
[0001.158] Switching to 800000 KHz Success
[0001.164] DT Write: emc-table@40800 succeeded
[0001.170] DT Write: emc-table@68000 succeeded
[0001.176] DT Write: emc-table@102000 succeeded
[0001.183] DT Write: emc-table@204000 succeeded
[0001.189] DT Write: emc-table@408000 succeeded
[0001.195] DT Write: emc-table@665600 succeeded
[0001.202] DT Write: emc-table@800000 succeeded
[0001.208] DT Write: emc-table@1065600 succeeded
[0001.215] DT Write: emc-table@1331200 succeeded
[0001.221] DT Write: emc-table@1600000 succeeded
[0001.225] LPDDR4 Training: Write DT: Number of tables = 10
U-Boot 2015.07-rc2-g78c8468 (Nov 09 2016 - 19:39:36 -0800)
TEGRA210
Model: NVIDIA P2371-2180
DRAM: 4 GiB
C: Tegra SD/MMC: 0, Tegra SD/MMC: 1
*** Warning - bad CRC, using default environment
tegra-pcie: PCI regions:
tegra-pcie: I/O: 0x0000000012000000-0x0000000012010000
tegra-pcie: non-prefetchable memory: 0x0000000013000000-0x0000000020000000
tegra-pcie: prefetchable memory: 0x0000000020000000-0x0000000040000000
tegra-pcie: 4x1, 1x1 configuration
tegra-pcie: probing port 0, using 4 lanes
tegra-pcie: link 0 down, retrying
tegra-pcie: link 0 down, retrying
tegra-pcie: link 0 down, retrying
tegra-pcie: link 0 down, ignoring
tegra-pcie: probing port 1, using 1 lanes
tegra-pcie: link 1 down, retrying
tegra-pcie: link 1 down, retrying
tegra-pcie: link 1 down, retrying
tegra-pcie: link 1 down, ignoring
In: serial
Out: serial
Err: serial
Net: No ethernet found.
Hit any key to stop autoboot: 2 1 0
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1…
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1…
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
977 bytes read in 106 ms (8.8 KiB/s)
p2371-2180 eMMC boot options
1: primary kernel
Enter choice: 1: primary kernel
Retrieving file: /boot/initrd
6902654 bytes read in 213 ms (30.9 MiB/s)
Retrieving file: /boot/Image
20048144 bytes read in 505 ms (37.9 MiB/s)
append: fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff2bf000 nvdumper_reserved=0xff23f000 core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 net.ifnames=0 root=/dev/mmcblk0p1 rw rootwait
Retrieving file: /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
423986 bytes read in 394 ms (1 MiB/s)
Flattened Device Tree blob at 82000000
Booting using the fdt blob at 0x82000000
reserving fdt memory region: addr=80000000 size=20000
Using Device Tree in place at 0000000082000000, end 000000008206a831
ERROR: Cannot read board ID EEPROM
WARNING: failed to read board EEPROM
Starting kernel …
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.10.96-tegra (buildbrain@mobile-u64-1072) (gcc version 4.8.2 (GCC) ) #1 SMP PREEMPT Wed Nov 9 19:42:57 PST 2016
Most of this is standard to Linux in general, and not necessarily Ubuntu or a Jetson. The basic process is to load a kernel, and the kernel then loads init…init is process ID 1, and basically owns everything (only the kernel owns PID 1). Init is designed to bring up everything else, and on modern systems, this is the “systemd” software (on command line typically manipulated with “systemctl”). So the steps you are looking at are orchestrated by systemd (init), and most of customization implies knowing how to use and change system services via systemctl and XML file edits (both change systemd).
An example is that different run levels are defined to use or not use graphics…this depends on which Linux distribution you look at for details, but often you get boot in the order of single user mode, multi-user mode, and then graphical mode. Single user mode is often not seen except on a rescue disk. Text only is multi-user mode…similar to rescue mode except it has login management services for user names and passwords. What you’re interested in is quickly reaching what is essentially a “bookmark” in run levels known by systemctl as “multi-user.target”. Brief note: Slightly older systems do not use systemd, but instead just init scripts written in bash; there are cases of legacy software still using init scripts only and not being managed by systemctl, but on Ubuntu 16.04 I doubt you’ll run into this.
One catch about knowing if you have reached multi-user.target or not is that video has to function…unless your access is by means which don’t require video. That would be serial console. If you want to see the times it takes to get somewhere and segregate video setup time from actually reaching that state you should set up a serial console (serial console even shows up during U-Boot before a kernel ever loads). See:
http://elinux.org/Jetson/TX1_Serial_Console
Video itself has to be set up whether in text mode or graphics mode…text mode just has more standard modes which are common. Regardless, there is a query in modern monitors (everything except 15-pin VGA D-sub connectors) for the video card and video software to query a monitor for what its capabilities are (“EDID”). There will be a blank screen until the video system sets a mode which the monitor can work with…if there are complications with EDID…especially if EDID fails and something must time out before falling back to a default mode…then video will take several seconds longer to show up. Basically I’m thinking your next step would be to monitor a serial console and see if a serial console login is occurring faster than video is being configured…in which case there may be steps to change video setup.
There may be other services which are started by systemd (init) which you do not use. Multi-user.target is really a bookmark of many other services and requirements, so it is possible you can reduce what sub-targets are run. Although you could directly modify systemd files, this is not the correct approach. If you look at “/etc/systemd/system/”, this directory overrides files of the same name in the actual systemd lib directories. A symbolic link from “/etc/systemd/system/” to a systemd lib file just says to do exactly what is in that bookmark…copying that file instead and editing that file serves as an override mechanism without being invasive to the default Ubuntu install. You may need to learn about the XML configuration of these files to effectively remove what you do not want.
Note that on command line not all Jetson targets behave exactly as the desktop version. On a desktop these would flip between multi-user.target and graphical.target, and may initialize something on a Jetson without doing everything exactly as a desktop would:
sudo systemctl isolate mutli-user.target
sudo systemctl isolate graphical.target
On the kernel command line some environment may be passed directly to systemd. An example for a desktop host to force boot directly to text mode without graphics (this may not work correctly on a Jetson…it certainly would not work on older systems using init files instead of systemd):
systemd.unit=multi-user.target
So what you want to be specified from the moment the kernel is loaded is via the kernel “APPEND” parameter in extlinux.conf of a systemd.unit. The rest is via edits of the XML files you copy from somewhere in “/lib/systemd/system/” to “/etc/systemd/system/”. Video config is a different topic, you’ll have to find out first if video is delaying, or if it is just taking more time than you want to get to multi-user.target.
One comment on disabling graphical mode: There is a difference between going from multi-user.target to graphical.target and then not loading anything, followed by falling back to text mode, versus going to mutli-user.target and never attempting graphical.target. The former case takes the time to go through all of the video mode changes and software load and fail, while the latter case just configures text-mode and never wastes time trying graphical setup when you know graphics is not desired.
Thanks for the detailed response. Unfortunately what I am looking for is the TX-1/U-Boot specific stuff that happens BEFORE getting to “Starting Kernel”. It currently takes about 10 seconds from reset to kernel start and I’d love to get that down to <5 which seems do-able by getting rid of the PCI probe and user input waits. Just hoping somebody has already done this and found any gotchas.
You may find this of use:
http://http.download.nvidia.com/tegra-public-appnotes/t210-nvtboot-flow.html
Once you get to U-Boot itself it reads “/boot/extlinux/extlinux.conf”, which is where it gets configuration which runs on top of the environment variables within U-Boot itself. Serial console once again helps here since you can see time taken to reach U-Boot and time spent actually in U-Boot. I’m not sure of order of device probes, but serial console might provide clues as to whether this is in nvtboot or U-Boot. I imagine it is possible to edit the U-Boot source or configuration and put a custom U-Boot in which eliminates some of the steps, but you might be surprised at how much can be eliminated by editing environment variables within U-Boot via serial console (many of these are macros which execute searches for available devices, so on…but if you already know which device you want, then you can edit out probing for other devices). Serial console is your friend! :)
In the boot process, u-boot has an interactive mode for you to choose, from serial console, which system you want to start in case you have several ones.
It also has a default system to run, and a timeout after which it runs the default system. You are probably seeing this timout to expire.
There was a u-boot environment variable bootdelay or timeout that you could set to a lower value for testing, or 0 to disable interactive mode.
Hi Sperok ,
Did you resolve the boot time issue ? Did you get below 5 seconds ? How ?
Thanks,
ranran