Native boot from M.2 NVME SSD

Does it boot normally with just the sdcard?

yes; once it is flashed with sdkmanager to sdcard it boots from sdcard normally

so for gathering the debug logs ; the J14 [UART RT, TX pinsunder the module] of NX Jetson gets connected to pins 8,10 of the AGX 40 pin header, right?
then NX withou sdcard but with nvme gets booting up to the point when the logo is shown, then at AGX Jetson the outputs will be shown e.g. with minicom or picocom listening to one of /dev/tty ports?

Look at the flash log it seems that the flash process succeed but my guess is that Cboot cannot find NVMe for some reason. You would need an serial cable like this

to get the serial log.

Another option is to try another NVMe if you have a spare.

I have this


I guess it can be connected from NX UART RX,TX of J14 to AGX JEtson 40 pin header UART pins 8,10 instead of procuring the usb to serial adapter. right?

@lhoang

I think so. You can try it should be ttyTHS0 on jetson xavier.

it shows empty line, unless I power off the Nx in which case it starts to print symbols:

 cat /dev/ttyTHS0
����������������������������������������������������������������������������������������F�������

grey seems pin 4 [ TX UART] on [J14/NX] it is connected to pin10 on 40 pin header on AGX
red is connected to pin 8 on AGX of 40 pin header; on Nx it is conencted to pin 3 of J14

I think you need to use something like

minicom -D /dev/ttyTHS0

minicom shows welcome scxreen on AGX:

Welcome to minicom 2.7.1

OPTIONS: I18n 
Compiled on Aug 13 2017, 15:25:34.
Port /dev/ttyTHS0, 14:00:11

Press CTRL-A Z for help on special keys

However after NX is powered off it will axd extra symbol:

&Û۶þ

after some attempts it shows

Compiled on Aug 13 2017, 15:25:34.
Port /dev/ttyTHS0, 14:00:11

Press CTRL-A Z for help on special keys
                                                                               p[0007.289] I> tegra_pcie_info[5].cfg0_base = 0x3a000000“²’²¶·¶¶ò’2&$򐒲&766vv²’€Ë[0007.290] I> tegra_pcie_info[5].cfg1_base = 0x3a020000
[0007.290] I> tegra_pcie_info[5].atu_dma_base = 0x3a040000
[0007.291] I> tegra_pcie_bus[5].mem = 0x3a200000
[0007.291] I> Scanning busnr: 0 devfn: 0
[0007.291] I> PCIe IDs: 0x1ad010de
[0007.294] I> PCIe RID_CC: 0x60400a1
[0007.298] I> Scanning busnr: 1 devfn: 0
[0007.314] I> PCIe IDs: 0xffffffff
[0007.315] I> Number of PCIe devices detected: 1
[0007.315] E> tegrabl_nvme_init: Failed tegrabl_pcie_get_dev(5); error=0x0
[0007.316] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800612
[0007.322] W> Failed to open NVME-5, err = 80800612
ing kernel-bootctrl from partition device 10-5
[0007.345] E> Cannot find partition kernel-bootctrl                             
[0007.350] E> Cannot open partition ke not initialized                          
[0007.399] W> Boot logo display failed...                                       
[0007.403] I> Kernel EP: 0x0, DTB: 0x0                                          
[0007.407]                                                                      
[0007.408] --> pc:0xA060EB00, sp:0xA06A41F0                                     
[0007.457] => pc:0xA060EAD4, sp:0xA06A4200                                      
[0007.461] ----------------------------------------x23 0x               0       
[0007.523] x24 0x               0 x25 0x               0 x26 0x               0

another try

$“’’’³¶····¶6¶ù22¶w76&» €öö62�����¶··¶¶»û·²62û’�����²6¶·¶²²“ÒöҶ²·¶¶·¶¶³"²‘’:[0006.733] I> APPL initialization ...
[0006.737] I> poweron phys
[0006.739] I> tegrfo[4].cfg1_base = 0x36020000
[0006.980] I> tegra_pcie_info[4].atu_dma_base = 0x36040000
[0006.981] I> tegra_pcie_bus[4].mem =grabl_car_rst_set(APB, 5) ...
[0007.091] I> tegrabl_car_clk_enable(5) ...
[0007.095] I> tegrabl_car_rst_clear(APB, 5) ...
[0fo[5].cfg1_base = 0x3a020000
[0007.292] I> tegra_pcie_info[5].atu_dma_base = 0x3a040000
[0007.293] I> tegra_pcie_bus[5].mem =              40 x14 0x               1 x10
[0007.507] x16 0x               9 x17 0x               1 x18 0x

another try


Terminal ready
�vwv6676�����������66�fv6��������6�3&22������22���2����������"22�2�2�"2����������22���"������������"������6��6"���26��w����r�"2���22�6v���6&���v6�f����@�����'6���w666����"2�������"�����������fv&7�76�չ��j�э��
                                                                             с��Ţ�
  ����j�����r����J����Ʌ��}��ݕ�}��}������ݕɁz��������*�����j�����r����J����Ʌ��}��ݕ�}��}������ݕɁz�������*���������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������*���������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������*���������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������*���������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������2���������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������2ł�������r����J����Ʌ��}��ݕ�}��}������ݕɁz�������2ɂ����
           [0007.331] E> NVME (5) boot failed, err: 0x80800612
[0007.336] I> ########## Fixed storage boot ##########
[0007.340] I> Loadrnel-bootctrl
[0007.354] W> tegrabl_get_kernel_bootctrl: failed to read primary bootctrl data
[0007.362] I> Loading kernel-bootctrl_b from partition
[0007.366] E> Cannot find partition kernel-bootctrl_b
[0007.371] E> Cannot open partition kernel-bootctrl_b
[0007.376] W> tegrabl_get_kernel_bootctrl: failed to read recovery bootctrl data
[0007.383] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0007.390] E> Blockdev open: exit error
[0007.394] E> tegrabl_display_clear: display is---------------------------------------------
[0007.413] Synchronous Exception: UNKNOWN EXCEPTION
[0007.418] -----------------------------------------------
[0007.423] 
[0007.424] ESR 0x2000000: ec 0x0, il 0x1, iss 0x0
[0007.429] -----------------------------------------------
[0007.434]  [Stack Trace]
[0007.436] 
[0007.437] => pc:0x00000000, sp:0xA06A3F30
[0007.441] => pc:0xA060F790, sp:0xA06A4160
[0007.445] => pc:0xA060F7A4, sp:0xA06A41B0
[0007.449] => pc:0xA060F58C, sp:0xA06A41E0
[0007.453] =-------
[0007.466] iframe 0xa06a3e40:
[0007.468] x0  0x               0 x1  0x               0 x2  0x               0 x3  0x               0
[0007.478] x4  0x               0 x5  0x              20 x6  0x         b200123 x7  0x        ffffffc0
[0007.487] x8  0x               0 x9  0xffffffffffffffff x10 0x               6 x11 0x               2
[0007.496] x12 0x               1 x13 0x              40 x14 0x               1 x15 0x             2c0
[0007.505] x16 0x               9 x17 0x               1 x18 0x               0 x19 0x               0
[0007.514] x20 0x               0 x21 0x               0 x22 0x               0 
                                                                               [0007.532] x28 0x               0 x29 0x        a06a4160 lr  0x        a060f744 sp  0x        a06a3f30
[0007.541] elr 0x               0
[0007.544] spsr 0x        400003c9
[0007.548] -----------------------------------------------
[0007.553] panic (caller 0xa0601238): die
[0007.557] HALT: spinning forever...
6:�v7��66f��2�6�������6��������������22����������66&6�7�7�6������������"�2������w6�6����������� ��$��66�������&��3������������666&2�����������2�2����������6v�$�2&r*6v�2�����26���������2��&���2273�������22226�����6����������22���������2�r2���$��Ӳ��������22666�"2266w����ֶ�������26���~3��������������������������������6����22"66����2��w��$��������������3322������������2"2����22���&726��������������6&%����26�v6��6�����2��� 	�������732����6�������������3��6vff���������������6&"���������6�����3���62���6�~��"����������������������abl_locate_pcie_ctrl_in_dt: found match at 0x14140000
[0006.745] I> tegrabl_power_on_phy: controller 3 not available
[0006.751] E> Failed to power on phy on controller-3
[0006.756] W> Failed tegrabl_pcie_soc_init(), error=0x1
[0006.761] I> Failed to initialize SoC Host PCIe controller
[0006.766] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(3); error=0x1
[0006.772] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800601
[0006.779] W> Failed to open NVME-3, err = 80800601
[0006.784] W> Failed to initialize device 10-3
[0006.788] E> NVME (3) boot failed, err: 0x80800601
[0006.792] I> ########## NVME (4) boot ##########
[0006.797] I> Initializing nvme device instance 4
[0006.801] I> Initializing nvme controller
[0006.806] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14160000
[0006.812] I> vpcie3v3-supply not found
[0006.815] I> vpcie12v-supply not found
[0006.819] W> Failed to get nvidia,plat-gpios
[0006.823] I> tegrabl_pcie_soc_preinit: (4):
[0006.827] I> Unpowergate
[0006.830] I> tegrabl_car_clk_disable(4) ...
[0006.833] I> tegrabl_car_rst_set(CORE, 4) ...
[0006.838] I> tegrabl_car_rst_set(APB, 4) ...
[0006.842] I> tegrabl_car_clk_enable(4) ...
[0006.846] I> tegrabl_car_rst_clear(APB, 4) ...
[0006.850] I> tegrabl_set_ctrl_state(4)
[0006.853] I> CLR PCIE_APB:6
[0006.856] I> tegrabl_pcie_soc_init: (4):
[0006.860] I> APPL initialization ...
[0006.863] I> poweron phys
[0006.866] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14160000
[0006.872] I> tegrabl_power_on_phy: power on phy @0x3f40000
[0006.979] I> PCIe controller-4 link is up
[0006.980] I> tegra_pcie_info[4].cfg0_base = 0x36000000
[0006.980] I> tegra_pcie_in 0x36200000
[0006.981] I> Scanning busnr: 0 devfn: 0
[0006.981] I> PCIe IDs: 0x1ad110de
[0006.985] I> PCIe RID_CC: 0x60400a1
[0006.988] I> Scanning busnr: 1 devfn: 0
[0006.992] I> PCIe IDs: 0xc82210ec
[0006.995] I> PCIe RID_CC: 0x2800000
[0006.998] I> PCI Config: I/O=0x36100000, Memory=0x36200000
[0007.003] I> IO bar_num=0 bar=0x36100000
[0007.007] I> MEM64 bar_num=2 bar=0x36200000
[0007.011] I> Number of PCIe devices detected: 2
[0007.016] E> tegrabl_nvme_init: Failed tegrabl_pcie_get_dev(4); error=0x0
[0007.023] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800612
[0007.029] W> Failed to open NVME-4, err = 80800612
[0007.033] W> Failed to initialize device 10-4
[0007.038] E> NVME (4) boot failed, err: 0x80800612
[0007.042] I> ########## NVME (5) boot ##########
[0007.047] I> Initializing nvme device instance 5
[0007.051] I> Initializing nvme controller
[0007.056] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x141a0000
[0007.061] I> vpcie3v3-supply not found
[0007.065] I> vpcie12v-supply not found
[0007.069] W> Failed to get nvidia,plat-gpios
[0007.073] I> tegrabl_pcie_soc_preinit: (5):
[0007.077] I> Unpowergate
[0007.079] I> tegrabl_car_clk_disable(5) ...
[0007.083] I> tegrabl_car_rst_set(CORE, 5) ...
[0007.087] I> te007.100] I> tegrabl_set_ctrl_state(5)
[0007.103] I> CLR PCIE_APB:6
[0007.106] I> tegrabl_pcie_soc_init: (5):
[0007.110] I> APPL initialization ...
[0007.113] I> poweron phys
[0007.116] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x141a0000
[0007.122] I> tegrabl_power_on_phy: power on phy @0x3eb0000
[0007.127] I> tegrabl_power_on_phy: power on phy @0x3ec0000
[0007.133] I> tegrabl_power_on_phy: power on phy @0x3ed0000
[0007.138] I> tegrabl_power_on_phy: power on phy @0x3ee0000
[0007.143] I> tegrabl_power_on_phy: power on phy @0x3ef0000
[0007.149] I> tegrabl_power_on_phy: power on phy @0x3f00000
[0007.154] I> tegrabl_power_on_phy: power on phy @0x3f10000
[0007.159] I> tegrabl_power_on_phy: power on phy @0x3f20000
[0007.291] I> PCIe controller-5 link is up
[0007.291] I> tegra_pcie_info[5].cfg0_base = 0x3a000000
[0007.292] I> tegra_pcie_in 0x3a200000
[0007.293] I> Scanning busnr: 0 devfn: 0
[0007.293] I> PCIe IDs: 0x1ad010de
[0007.296] I> PCIe RID_CC: 0x60400a1
[0007.300] I> Scanning busnr: 1 devfn: 0
[0007.316] I> PCIe IDs: 0xffffffff
[0007.317] I> Number of PCIe devices detected: 1
[0007.317] E> tegrabl_nvme_init: Failed tegrabl_pcie_get_dev(5); error=0x0
[0007.318] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800612
[0007.324] W> Failed to open NVME-5, err = 80800612
[0007.329] W> Failed to initialize device 10-5
[0007.333] E> NVME (5) boot failed, err: 0x80800612
[0007.338] I> ########## Fixed storage boot ##########
[0007.342] I> Loading kernel-bootctrl from partition
[0007.347] E> Cannot find partition kernel-bootctrl
[0007.352] E> Cannot open partition kernel-bootctrl
[0007.356] W> tegrabl_get_kernel_bootctrl: failed to read primary bootctrl data
[0007.364] I> Loading kernel-bootctrl_b from partition
[0007.368] E> Cannot find partition kernel-bootctrl_b
[0007.373] E> Cannot open partition kernel-bootctrl_b
[0007.378] W> tegrabl_get_kernel_bootctrl: failed to read recovery bootctrl data
[0007.385] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0007.392] E> Blockdev open: exit error
[0007.396] E> tegrabl_display_clear: display is not initialized
[0007.401] W> Boot logo display failed...
[0007.405] I> Kernel EP: 0x0, DTB: 0x0
[0007.409] 
[0007.410] -----------------------------------------------
[0007.415] Synchronous Exception: UNKNOWN EXCEPTION
[0007.420] -----------------------------------------------
[0007.425] 
[0007.426] ESR 0x2000000: ec 0x0, il 0x1, iss 0x0
[0007.431] -----------------------------------------------
[0007.436]  [Stack Trace]
[0007.438] 
[0007.439] => pc:0x00000000, sp:0xA06A3F30
[0007.443] => pc:0xA060F790, sp:0xA06A4160
[0007.447] => pc:0xA060F7A4, sp:0xA06A41B0
[0007.451] => pc:0xA060F58C, sp:0xA06A41E0
[0007.455] => pc:0xA060EB00, sp:0xA06A41F0
[0007.459] => pc:0xA060EAD4, sp:0xA06A4200
[0007.463] -----------------------------------------------
[0007.468] iframe 0xa06a3e40:
[0007.470] x0  0x               0 x1  0x               0 x2  0x               0 x3  0x               0
[0007.480] x4  0x               0 x5  0x              20 x6  0x         b200123 x7  0x        ffffffc0
[0007.489] x8  0x               0 x9  0xffffffffffffffff x10 0x               6 x11 0x               2
[0007.498] x12 0x               1 x13 0x               0 x19 0x               0
[0007.516] x20 0x               0 x21 0x               0 x22 0x               0 x23 0x               0
[0007.525] x24 0x               0 x25 0x               0 x26 0x               0 x27 0x               0
[0007.534] x28 0x               0 x29 0x        a06a4160 lr  0x        a060f744 sp  0x        a06a3f30
[0007.543] elr 0x               0
[0007.546] spsr 0x        400003c9
[0007.550] -----------------------------------------------
[0007.555] panic (caller 0xa0601238): die
[0007.558] HALT: spinning forever...
�����������������������������������������������������

Yeah, cboot cannot initialize your NVMe ssd for some reason. If you can try with another SSD it would be great.

Also let me take a look in the driver to see why this error happens.

it is with
Samsung 980 PRO PCIe 4.0 NVMe
I will try to see if I could release other gen3 ssd, but it is full with data

Oh I see it is Gen 4. We only tested it with NVMe Gen 3.

@lhoang
When gen4 initrd support will be available/tested?

I will need to discuss this internally. There is no official plan for now.

I had to move 1Tb over network to try with gen3 that worked but caused more inconviniences than solutions.
Is there any update on how to implement gen4 support?

@lhoang to clarify gen4 works on nx unless with initrdboot
e.g. it works with sdcard boot nvme solution shared earlier by Jetsonhacks as rootatnvmeSSD
but specifically initrd thing won’t work with gen4

@lhoang I got AGX to boot initrd from gen4
Any updte regarding nx gen4 support?

agx has emmc already inside so it would work even if cboot can not boot from NVMe as it could boot from emmc and use NVMe as file system; similar to nx can boot from sd and use nvme as file system.

if you are ok with sd card inside nx, you can use the following command for initrd flash:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_external.xml jetson-xavier-nx-devkit nvme0n1p1

for your nvme gen4, we are currently investigating it.

@lhoang Thank you for following up!
on AGX I just flashed from the host with

sudo ./nvsdkmanager_flash.sh --storage nvme0n1p1 

doesn’t it mean that eMMC is not used at all?
for AGX I used the same gen4 ssd that did not work on NX previously.

Could you also extend if the initrd could be flashed remotely from a network to Jetson? The documentation points to flashing with initrd from flashing from NFS

Flashing from NFS
Applies to: Jetson Xavier NX and Jetson AGX Xavier series only
Note
Flashing from NFS is deprecated. Use Flashing with initrd instead.