Flash Failure (Failed to Mount Kernel Configuration File System)

In trying to flash an Aetina carrier board (Jetson Xavier NX devkit) that we have in order to use with JetPack 5.1.4/L4T35.6.0, I encountered some errors during the process.

I use the L4T and sample root filesystem files from https://developer.nvidia.com/embedded/jetson-linux-r3560 and the only additions that I make are using the script to create a default username/password/autologin, removing line 117 of the ota_make_recovery_img_dtb.sh since our version of openSSL doesn’t work with dsa keys anymore, and a custom uefi_jetson.bin file that just has the logo replaced, which has worked in past flash attempts with other boards.

I’ve done this process many times before and it usually works to flash the eMMC first with:

sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

And then flashing the NVMe so it becomes the boot drive with:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme.xml -S 900GiB --external-only --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

I had used these files before to flash a different Jetson a few weeks ago, and it was successful. However, this time, after flashing the NVMe, the system showed the custom uefi boot logo like normal, and then the display had black/grey pixels gradually fill up the screen from top to bottom. I thought the flash may have gone wrong during the process and tried again, but it happened again. I deleted the whole L4T 35.6.0 directory I made, made a new one, unzipped the files provided on the website above, and flashed again, but the same result occurred.

I then took the NVMe from the previously successfully flashed Jetson and put it in the current problematic one, but the result was the same. Removing the NVMe and trying with just eMMC did have it boot to the desktop successfully and showed the Ubuntu welcome window that appears after logging in for the first time.

Putting the NVMe back in gave the same black screen wipe as mentioned above. Rebooting again without the NVMe also made the eMMC-only boot get the screen wipe issue too. Flashing again after that did have the dmesg output show up after booting, but I get many errors related to mounting.

Failed to mount Kernel Debug File System
Failed to mount Kernel Trace File System
Failed to mount RPC Pipe File System
Failed to start Remount Root and Kernel File System
Failed to Mount Kernel Configuration File System
Failed to Mount Arbitrary Executable File Formats File System

I then tried an L4T 35.4.1 flash that worked in the past, but the result was the same. The first few flashes were done with a long USB cable, so I tried again with a short one but nothing changed. Before flashing, the board worked normally, so my only guess at this point was an ESD-related damage issue, but if anyone has any ideas, I would appreciate the help. We have other Jetsons that I could try to flash as well to see if it’s just this one, but I’m a bit concerned of the outcome at this point if this could be caused by potentially the cable or PC that we’re using to flash.

As an update, it turns out the Jetson was flashed successfully every time. The issue seems to stem from the display that we use. It is 1024x600, so that likely caused some sort of initialization issue on startup. The reason why I hadn’t noticed it with the previous flashes was because that Jetson was connected to a standard HD monitor. If the Jetson boots to the standard monitor and then I connect the HDMI to the small touch panel, the display works as intended. But if it boots to the touch panel, then the display initialization seems to have failed and even switching it to the standard monitor afterwards does not give any screen display. I can look into the display drivers for now to see if the issue can be resolved, but if anyone has any information as to how this may be fixed, I would appreciate any help, thank you.

Hi joseph.zc,

It seems your issue is not relating to flash itself but display issue.
Do you mean it works as expected if you use normal HD display?

Could you share the full serial console log during boot up for further check?
UEFI does support only the following 3 resolutions currently.

Found 640x480
Found 1280x720
Found 1920x1080

The bmp that I build into the custom UEFI is 640x480, and on previous JetPack releases it is displayed normally. It doesn’t fill the screen but is displayed at the bottom center. This still occurs without the modified uefi_jetson.bin file though.

Here is what dmesg shows (with grep display) on boot with the small display (these are not shown when connected to a 1080p display):

[    0.998065] tegradccommon 15200000.display-hub: Nvhost Channel map failed
[    5.847359] tegradccommon 15200000.display-hub: host1x channel mapped
[    5.847605] tegradccommon 15200000.display-hub: dc_common syncpt # 1 allocated
[    5.848011] tegradc 15200000.display: Adding to iommu group 26
[    5.850106] tegradc 15210000.display: Adding to iommu group 26
[    5.854387] tegradccommon 15200000.display-hub: dma mapping done
[    6.184861] tegradc 15200000.display: disp0 connected to head0->sor1
[    6.199849] tegradc 15200000.display: DT parsed successfully
[    6.214011] tegradc 15200000.display: Display dc.00000000895ae0e5 registered with id=0
[    6.250020] tegradc 15200000.display: vblank syncpt # 8 for dc 0
[    6.264519] tegradc 15200000.display: vpulse3 syncpt # 9 for dc 0
[    6.328674] tegradc 15200000.display: hdmi: invalid prod list prod_list_hdmi_board
[    6.334440] tegradc 15200000.display: hdmi: tegra_hdmi_tmds_range_read(bd) failed
[    6.369454] tegradc 15200000.display: probed
[    6.415291] tegradc 15200000.display: fb registered
[    6.499455] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.499669] tegradc 15200000.display: timeout waiting for postcomp init state to promote
[    6.551443] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.561328] tegradc 15200000.display: timeout waiting for win assignments to promote
[    6.561334] tegradc 15200000.display: tegra_nvdisp_head_enable, failed head enable
[    6.561639] tegradc 15210000.display: disp1 connected to head1->sor
[    6.575645] tegradc 15210000.display: parse_dp_settings: No dp-lt-settings node
[    6.589471] tegradc 15210000.display: DT parsed successfully
[    6.602601] tegradc 15210000.display: Display dc.00000000087fe225 registered with id=1
[    6.613277] tegradc 15210000.display: vblank syncpt # 14 for dc 1
[    6.627465] tegradc 15210000.display: vpulse3 syncpt # 15 for dc 1
[    6.641350] tegradc 15210000.display: probed
[    6.722551] tegradc 15210000.display: fb registered
[    6.722565] tegradc 15200000.display: blank - powerdown
[    6.726896] tegradc 15200000.display: unblank
[    6.779436] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.779441] tegradc 15200000.display: dc timeout waiting for DC to stop
[    6.831436] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.831441] tegradc 15200000.display: dc timeout waiting for DC to stop
[    6.883440] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.883446] tegradc 15200000.display: timeout waiting for postcomp init state to promote
[    6.935439] tegradc 15200000.display: dc_poll_register 0x41: timeout
[    6.935445] tegradc 15200000.display: timeout waiting for win assignments to promote
[    6.935449] tegradc 15200000.display: tegra_nvdisp_head_enable, failed head enable
[    6.935473] tegradc 15200000.display: update windows ret = -14
[    6.935477] tegradc 15200000.display: sync windows ret = -14
[    7.015128] tegradc 15210000.display: blank - powerdown
[    7.067704] tegradc 15200000.display: hdmi: plugged
[   16.700732] tegradc 15200000.display: blank - powerdown
[   16.700907] tegradc 15200000.display: unblank
[   16.751498] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   16.751506] tegradc 15200000.display: dc timeout waiting for DC to stop
[   16.803471] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   16.803479] tegradc 15200000.display: dc timeout waiting for DC to stop
[   16.855552] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   16.855568] tegradc 15200000.display: timeout waiting for postcomp init state to promote
[   16.907471] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   16.907478] tegradc 15200000.display: timeout waiting for win assignments to promote
[   16.907485] tegradc 15200000.display: tegra_nvdisp_head_enable, failed head enable
[   16.907515] tegradc 15200000.display: update windows ret = -14
[   16.907523] tegradc 15200000.display: sync windows ret = -14
[   19.143783] tegradc 15200000.display: blank - powerdown
[   19.143840] tegradc 15200000.display: unblank
[   19.195531] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.195547] tegradc 15200000.display: dc timeout waiting for DC to stop
[   19.247498] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.247506] tegradc 15200000.display: dc timeout waiting for DC to stop
[   19.299478] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.299487] tegradc 15200000.display: timeout waiting for postcomp init state to promote
[   19.351509] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.351517] tegradc 15200000.display: timeout waiting for win assignments to promote
[   19.351523] tegradc 15200000.display: tegra_nvdisp_head_enable, failed head enable
[   19.353071] tegradc 15200000.display: unblank
[   19.403485] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.403493] tegradc 15200000.display: dc timeout waiting for DC to stop
[   19.455496] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.455505] tegradc 15200000.display: dc timeout waiting for DC to stop
[   19.507575] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.507589] tegradc 15200000.display: timeout waiting for postcomp init state to promote
[   19.559476] tegradc 15200000.display: dc_poll_register 0x41: timeout
[   19.559484] tegradc 15200000.display: timeout waiting for win assignments to promote
[   19.559491] tegradc 15200000.display: tegra_nvdisp_head_enable, failed head enable
[   19.561129] tegradc 15210000.display: blank - powerdown
[  341.568241] tegradc 15200000.display: blank - powerdown
[  341.568260] tegradc 15210000.display: blank - powerdown

You should replace 3 bmp files if you want to use custom boot logo.
Would it works as expected before you update the custom bmp?

What’s the release of “previous JetPack releases”?

Have you referred to Jetson Xavier platform specific configurations — NVIDIA Jetson Linux Developer Guide 1 documentation to configure the device tree for your display?

Yes I replaced the 3 bmp images, but they have nothing to do with this issue since it happens with or without the modified UEFI file.

Previous JetPack releases that I’ve used were L4T 35.4.1 and 35.2.1, and 35.4.1 also used the custom UEFI without any issues before. The UEFI also shows the custom logo in this 35.6.0 setup, but the screen gets the error when it should be displaying the login image.

It looks like the issue might be what is described in the link you provided, but I can’t seem to get it to work. I tried using the script with the hactive and vactive set to 1024 and 600 respectively, but the issue still persisted after flashing.

./nv-enable-hard-coded-kernel-boot-display-mode.sh dtb/tegra194-p3668-0001-p3509-0000.dtb

Unfortunately I’m not too familiar with displays to know how to resolve this issue, but if there are other logs that I might be able to provide, let me know and I will try.

Okay, it seems you issue is relating to display driver rather than flash failed issue.
I would suggest you creating another topic and share the full serial console log/dmesg and device tree for further check.