Jetson Xavier NX how to fix “ERROR: mmcblk0p1 mount fail” boot up error

[hezhensheng007]

Hi KevinFFF，

Do you mean that you hit 5603 times errors out of 10000 times power cut-off test?
---->That’s not what I meant， the real means that is the time started to go wrong out of 5603 times power cut-off test .

Okay, please verify with the latest JP5.1.4(R35.6.0) and check if there’s any improvement.
And let us know if OverlayFS may meet your requirement in your use case.
---->Okay, I’ll try to verify it, but it will take long time.

[hezhensheng007]

@linuxdev

ut to illustrate do the following:

hi linuxdev,
Thanks for your guidance, I will communicate with the hardware engineer to add a supercapacitor for testing purposes，this will take some time，Thanks you so much！

[eexuke]

hi huangyongxi,
In my project, I came up with a perfect solution and it was proven in practice, all I needed to do was modify the l4t_initrd.img file in jetpack_SDK!

[eexuke]

hi， hezhensheng007。 In my project, I came up with a perfect solution and it was proven in practice, all I needed to do was modify the l4t_initrd.img file in jetpack_SDK!

[hezhensheng007]

hi eexuke，
can you show more detail ？ how to fix ？thanks a lot！

[eexuke]

hi， hezhensheng007，
Yes, of course.

1. First, when the rootfs is corrupted and the error “ERROR: mmcblk0p1 mount fail” is reported, the linux system will enter the initrd (init ramdisk). At this time, you can run the command “e2fsck /dev/mmcblk0p1” through NFS to repair the rootfs partition and restore the system.
2. So, based on the above principle, we find the file corresponding to initrd in jetpack is “Linux_for_tegra/bootloader/l4t_initrd.img”, and perform the following operations to modify it:

• sudo su
• mkdir ./tmp
• cp bootloader/l4t_initrd.img ./tmp/l4t_initrd.img
• mkdir ./tmp/temp && cd ./tmp/temp/
• gunzip -c …/l4t_initrd.img | cpio -i
• “modify initrd in the temp dir” : 1) copy e2fsck to temp/bin/， 2）modify temp/init file for auto run；
• find . | cpio -H newc -o | gzip -9 -n > …/my_l4t_initrd.img

3. Then replace bootloader/l4t_initrd.img with my_l4t_initrd.img
4. Through the above operations, we added the e2fsck command to the initrd and modified the self-boot script “init” to automatically execute e2fsck to fix mmcblk0p1 when mounting mmcblk0p1 failed.
5. In addition, you can execute “sudo dd if=/dev/zero of=/dev/mmcblk0p1 bs=1k seek=1 count=1” on the Jetson Xavier NX board to actively destroy the rootfs, so that the test can enter “ERROR: mmcblk0p1 mount fail” status.

Hope the above program helps you!

[hezhensheng007]

hi eexuke,
great, it is look great helpfull for me,I’ll verify it right away，Thanks so much！

[hezhensheng007]

hi eexuke，
i had tested your guidence and show it make effective, on development board with USB keyboad to run “e2fsck /dev/mmcblk0p1” when enter “ERROR: mmcblk0p1 mount fail” status，but there is not USB keyboard with my product in fact, do you hava any other suggestion? Thanks all！

[eexuke]

hi， hezhensheng007，
If you lead out the debug serial port, you can use the debug serial port to perform command operations, which function the same as the external keyboard;
In addition, you can debug the firmware on the development board and backup (replaced initrd), and then burn the perfect backup firmware into your product, thus solving the problem once and for all!

[hezhensheng007]

hi eexuke,
ok, got it, thanks you for your advise. i had try to use the debug serial port to perform command operations, which the devices can not get commands input from serial port，I don’t know why, It has been tested and RX/TX is ok when the nvidia jetson boot up normally.

[linuxdev]

I can’t go straight to any answer, but I will add these notes:

• There must be a USB driver for any USB in boot stages, and similarly, the serial UART driver must be present in boot stages for those stages to use the serial console. This is prior to a Linux kernel loading.
• Once an initrd runs it is in the Linux kernel. This implies a different USB and/or serial UART driver. Within the initrd those must be present either in the kernel Image file, or more commonly, in the kernel module. The kernel modules must be in the initrd, and not just on the eMMC. If the kernel modules are not present within the initrd, then USB and/or serial console would fail.
• It is most common to add kernel modules which are required for boot to the initrd; most other modules are left out. If it was unexpected to need the USB drivers and/or serial console drivers from the initrd, then this would explain an inability to use those functions within the initrd.
• In this case “USB” implies both “USB” and the USB “HID” drivers. The HID drivers are the actual “Human Interface Devices”, such as a keyboard or mouse. Although HID drivers always come with USB, and are used from USB, these are technically just keyboard/mouse drivers, and are not specifically USB other than the fact that it is always USB which pairs the HID drivers to USB input devices.

[hezhensheng007]

hi linuxdev，
Thank you for your reply!