After flash SPE firmware,enable GPIO-APP or SPI-APP ,the kernel and the SPE firmware cannot be started simultaneously

I am now trying to use the SPI application and have completed all the steps according to this Jetson Sensor Processing Engine (SPE) Developer Guide: SPI application (app/spi-app.c and app/spi-slv-app.c)
The issue I’m encountering now is that after I successfully flashed all the partitions, the Linux system on the development board fails to boot. I’d like to know why this is happening. Below is my log.
flash_log.txt (270.1 KB)
please help @jachen @WayneWWW @DavidDDD Thanks in advance

Can you also provide the UART log from device side?

br
Chenjian

boot_log.txt (36.0 KB)is this that u need?

I can see following error:
init uartc for combined uart
spi_test: Launching task…
> Task: Trigger mailbox for PSC-BL1 exit
I> Sending opcode 0x4d420802 to psc
eceived incorrect data
Received ACK from psc
I> Task: Start secure NOR provision
I> Skip Secure NOR provisioning
I> Task: Trigger load FSI keyblob
I> Task: Complete load FSI keyblob
I> Task: MB2-PSC_FW Key Manager Init
I> Sending opcode OP_PSC_KEY_MANAGER to psc-fw
I> Sending opcode 0x4b45594d to psc
wwdt_init: WDT boot cfg 0x710010 sts 0x10
bpmp: socket 0
bpmp: base binary md5 is da583751bbfe2b7f6e204562d97ff39e
bpmp: combined binary md5 is dc3fd3cac60f9b918e11d435834d12a3
bpmp: firmware tag is dc3fd3cac60f9b918e11-da583751bbf

FATAL ERROR [FILE=platform/top/bpmp/startup/sku.c, ERR_UID=598]: chip sku 0xd5 not in /sku/te980m_0

I’ve not got such issue before. Not sure whether the error comes from faulty SPE firmware, though that’s possible. Faulty SPE firmware may result in unexpected behavior.
Just a few debug suggestions.

  1. Restore the platform, i.e., re-flash the whole device with locally built SPE firmware WITHOUT ANY CHANGE, and make sure it boots well. Here, you can update the pinmux setting, or firewall setting, as “rt-aux-cpu-demo-fsp/doc/spi-app.md” describes, but keep the built SPE firmware with original source. After device’s booting, confirm the local built SPE firmware works well.
  2. Replace the SPE firmware with your own changes. Please refer to Flashing Support — NVIDIA Jetson Linux Developer Guide 1 documentation, and it will only update the SPE firmware partition.
  3. If it fails in step 2, you can make further debug, and focus on SPE firmware change.

br
Chenjian

Thank u for ur suggestion.I restore the platform, i.e., re-flash the whole device with locally built SPE firmware without an change,and it boots well,but get something error like this



boot.txt (46.2 KB)
I’m sure it can start up normally. I’ve decided to ignore this issue for now. Now I will update the pinmux setting or the firewall setting, as described in “rt-aux-cpu-demo-fsp/doc/spi-app.md,” and keep the built SPE firmware with the original source.

I have verified the following:

  1. First, I tested the device with the locally built SPE firmware WITHOUT ANY CHANGE, and it boots well.
  2. Then, I updated the pinmux setting or the firewall setting, as described in “rt-aux-cpu-demo-fsp/doc/spi-app.md,” and kept the built SPE firmware with the original source. After the device booted, the locally built SPE firmware worked well.

Additionally, I found that there are two tegra234-mb2-bct-scr-p3701-0000-override.dts files:

  • One is located at ${L4T}/bootloader/tegra234-mb2-bct-scr-p3701-0000-override.dts.
  • The other is located at ${L4T}/bootloader/generic/BCT/tegra234-mb2-bct-scr-p3701-0000-override.dts.

If I only modify the file at ${L4T}/bootloader/tegra234-mb2-bct-scr-p3701-0000-override.dts as instructed in the tutorial, everything reverts back after packaging. However, if I modify the file at ${L4T}/bootloader/generic/BCT/tegra234-mb2-bct-scr-p3701-0000-override.dts, the changes remain after packaging. I would like to know if this is intentional on your part.

Moreover, in the file soc/t23x/target_specific.mk, there is a line:

# 0 Builds for AGX Orin; 1 Builds for Orin Nano
ENABLE_SPE_FOR_ORIN_NANO := 0

Do I need to make changes here?

Additionally, I noticed that if I set:

# Enable = 1/Disable = 0 SPI sample app
ENABLE_SPI_APP := 1

the devkit fails to boot. However, if I change it back to 0, the devkit boots successfully. Could you please help me look into these issues?

Here are two UART serial logs for your reference. One is with SPI_APP=0,
flash-SPI_APP=0.txt|attachment (100.3 KB)
and this one can boot normally.
The other is with SPI_APP=1,
flash-SPI-APP=1.txt|attachment (87.7 KB)
and this one fails to boot. In this case, I have modified the content of spi-app.c as follows:
spi-app.c.txt (4.1 KB)
I would like to know why such issues are occurring and whether you have ever used the SPE application and SPI application?

From your log, it seems that the device hangs at initrd.
what’s the bsp version you are running? I checked 35.3.1 and 36.4, only one tegra234-mb2-bct-scr-p3701-0000-override.dts is located under ./bootloader/tegra234-mb2-bct-scr-p3701-0000-override.dts
Please double-confirm in your side.

For ENABLE_SPE_FOR_ORIN_NANO, it’s only related to GPIO-related code.

br
Chenjian

bsp version is 36.4.3
Sorry for the confusion. I meant to say that there are two tegra234-mb2-bct-scr-p3767-0000.dts files.
One is located at ${L4T}/bootloader/tegra234-mb2-bct-scr-p3767-0000.dts,
and the other is located at ${L4T}/bootloader/generic/BCT/tegra234-mb2-bct-scr-p3767-0000.dts.
If I only modify the file at tegra234-mb2-bct-scr-p3767-0000.dts as instructed in the tutorial, everything reverts back after packaging. However, if I modify the file at tegra234-mb2-bct-scr-p3767-0000.dts , the changes remain after packaging.

I noticed that if I set:

# Enable = 1/Disable = 0 GPIO sample app
ENABLE_GPIO_APP := 1

the kernel fails to boot. However, if I change it back to 0, the kernel boots successfully. Could you please help me look into these issues?
boot-GPIO-APP=1.txt|attachment (69.6 KB)
@jachen hi jachen,please help

Hello,

  1. for file tegra234-mb2-bct-scr-p3767-0000.dts, yes, you are right. The file ${L4T}/bootloader/generic/BCT/tegra234-mb2-bct-scr-p3767-0000.dts is the right one to be updated. Another one bootloader/tegra234-mb2-bct-scr-p3767-0000.dts is actually generated during flash procedure, and originated from the former one.
  2. If you enable GPIO app, please follow the guide in “rt-aux-cpu-demo-fsp/doc/gpio.md”. If GPIO interrupt is not mapped correctly, it will hang the system.

br
ChenJian

I have carefully checked every step, all of which were followed according to the instructions in “rt-aux-cpu-demo-fsp/doc/gpio.md”. I don’t quite understand what is meant by “GPIO interrupt is not mapped correctly”. Could you please explain it in more detail?

Please check following section, to make sure it’s updated correctly.
2. Update gpio interrupt mapping as below in ${L4T}/bootloader/generic/BCT/tegra234-mb1-bct-gpioint-p3767-0000.dts:

GPIO interrupt should route to AON R5 for GPIO test in SPE firmware.

I checked your log again, and it may not be the reason.
The system hangs after
[ 10.660463] Switching from initrd to actual rootfs
[ 10.664856] Kernel panic - not syncing:
[ 10.664860] Attempted to kill init! exitcode=0x00007e00

I’m a little confused. Generally, for SPE firmware update, it will not affect the root-FS switching. You can compare the kernel log with and without GPIO app.

br
Chenjian

boot_without_GPIO.txt (1.3 MB)
This is boot_log without GPIO

@jachen Hello, Jachen. This issue has been troubling us for a long time. Could you help us analyze it? Currently, after we launch the SPI APP or the GPIO APP, SPE and Linux cannot be started at the same time.

from the logs you uploaded, ( boot_without_GPIO.txt (1.3 MB) and boot-GPIO-APP=1.txt|attachment|attachment (69.6 KB)) I do not see the boot failure is related to SPE firmware.
I can see that all boot failure contains following log:
[ 10.630827] Rootfs mounted over PARTUUID=24d42b60-8186-460d-823d-af8cac79758f
[ 10.638807] Switching from initrd to actual rootfs
[ 10.643124] Kernel panic - not syncing:

And successful booting contains:
[ 7.414074] Rootfs mounted over PARTUUID=735eabc9-316b-4921-9502-560059a8e54c
[ 7.422408] Switching from initrd to actual rootfs
[ 7.467028] systemd[1]: System time before build time, advancing clock.
or
[ 10.047493] Rootfs mounted over PARTUUID=0c12c321-200f-4172-acaa-e265b5d63ae0
[ 10.055390] Switching from initrd to actual rootfs
[ 10.098789] systemd[1]: System time before build time, advancing clock.

Please check the storage device which contains root-FS.

br
ChenJian

When will the PARTUUID be changed? Our operation is merely to update and replace the SPE firmware into the file Linux_for_Tegra/bootloader/spe_t234.bin, without modifying the PARTUUID.

hi @jachen
boot-GPIO-APP=1 initrd_flash.txt|attachment (103.6 KB)
This file are the logs generated after I made modifications to the device tree and target_specific.mk following the guide in “rt-aux-cpu-demo-fsp/doc/gpio.md”, and then ran the command:

sudo BOARDID=3767 BOARDSKU=0005 FAB=TS4 ./tools/kernel_flash/l4t_initrd_flash.sh \
--external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" \
--showlogs --network usb0 jetson-orin-nano-devkit internal

Can you make some debug in your side?
I can see the command line in kernel:
[ 0.000000] Kernel command line: root=PARTUUID=24d42b60-8186-460d-823d-af8cac79758f rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 firmware_class.path=/etc/firmware fbcon=map:0 nospectre_bhb video=efifb:off console=tty0 bl_prof_dataptr=2031616@0x271E10000 bl_prof_ro_ptr=65536@0x271E00000

You can update the SPE firmware step by step.
For example, starting from SPI, and once it works, and go to next change.
In addition, L4T may switch to RCM boot, if it fails to boot after a few times.

br
Chenjian

What does the mean: “L4T may switch to RCM boot, if it fails to boot after a few times.”