Jetson Xavier NX, kernel panic at pc : tegra194_cbb_err_isr+0x19c/0x1b0

We use Jetson Xavier NX, Jetson Linux 35.3.1 part of JetPack 5.1.1

When boot kernel, we get following abnormal log:

[2024-05-15 07:09:42]  nvidia login: [   26.461840] tegradc 15200000.display: unblank
[2024-05-15 07:09:46]  [   26.464970] tegra_nvdisp_handle_pd_enable: Unpowergated Head0 pd
[2024-05-15 07:09:46]  [   26.465220] tegra_nvdisp_handle_pd_enable: Unpowergated Head1 pd
[2024-05-15 07:09:46]  [   26.491901] CPU:0, Error: cbb-noc@2300000, irq=15
[2024-05-15 07:09:46]  [   26.491908] **************************************
[2024-05-15 07:09:46]  [   26.491911] CPU:0, Error:cbb-noc
[2024-05-15 07:09:46]  [   26.491915] Error Logger: 0
[2024-05-15 07:09:46]  [   26.491928] ErrLog0: 0x80030000
[2024-05-15 07:09:46]  [   26.491931]   Transaction Type: RD  - Read, Incrementing
[2024-05-15 07:09:46]  [   26.491934]   Error Code: SLV
[2024-05-15 07:09:46]  [   26.491938]   Error Source: Target
[2024-05-15 07:09:46]  [   26.491944]   Error Description: Target error detected by CBB slave
[2024-05-15 07:09:46]  [   26.491956]   AXI2APB_5 bridge error: RDFIFOF - Read Response FIFO Full interrupt
[2024-05-15 07:09:46]  [   26.491963]   Packet header Lock: 0
[2024-05-15 07:09:46]  [   26.491966]   Packet header Len1: 3
[2024-05-15 07:09:46]  [   26.491970]   NOC protocol version: version >= 2.7
[2024-05-15 07:09:46]  [   26.491974] ErrLog1: 0x350e22
[2024-05-15 07:09:46]  [   26.491977] ErrLog2: 0x0
[2024-05-15 07:09:46]  [   26.491981]   RouteId: 0x350e22
[2024-05-15 07:09:46]  [   26.491985]   InitFlow: ccroc_p2ps/I/ccroc_p2ps
[2024-05-15 07:09:46]  [   26.491990]   Targflow: host1x_p2pm/T/host1x_p2pm
[2024-05-15 07:09:46]  [   26.491993]   TargSubRange: 7
[2024-05-15 07:09:46]  [   26.491997]   SeqId: 0
[2024-05-15 07:09:46]  [   26.492000] ErrLog3: 0x180
[2024-05-15 07:09:46]  [   26.492004] ErrLog4: 0x0
[2024-05-15 07:09:46]  [   26.492009]   Address accessed: 0x15200180
[2024-05-15 07:09:46]  [   26.492013] ErrLog5: 0x889f851
[2024-05-15 07:09:46]  [   26.492017]   Non-Modify: 0x1
[2024-05-15 07:09:46]  [   26.492020]   AXI ID: 0x11
[2024-05-15 07:09:46]  [   26.492024]   Master ID: CCPLEX
[2024-05-15 07:09:46]  [   26.492027]   Security Group(GRPSEC): 0x7e
[2024-05-15 07:09:46]  [   26.492031]   Cache: 0x1 -- Bufferable 
[2024-05-15 07:09:46]  [   26.492036]   Protection: 0x2 -- Unprivileged, Non-Secure, Data Access
[2024-05-15 07:09:46]  [   26.492040]   FALCONSEC: 0x0
[2024-05-15 07:09:46]  [   26.492043]   Virtual Queuing Channel(VQC): 0x0
[2024-05-15 07:09:46]  [   26.492048] **************************************
[2024-05-15 07:09:46]  [   26.492084] ------------[ cut here ]------------
[2024-05-15 07:09:46]  [   26.492088] kernel BUG at drivers/soc/tegra/cbb/tegra194-cbb.c:2057!
[2024-05-15 07:09:46]  [   26.492094] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[2024-05-15 07:09:46]  [   26.492100] Modules linked in: fuse lzo_rle lzo_compress zram ramoops reed_solomon realtek snd_soc_tegra210_ope snd_soc_tegra186_dspk snd_soc_tegra210_iqc snd_soc_tegra210_mvc snd_soc_tegra210_amx snd_soc_tegra210_dmic snd_soc_tegra186_asrc snd_soc_tegra186_arad snd_soc_tegra210_afc snd_soc_tegra210_adx snd_soc_tegra210_mixer snd_soc_tegra210_i2s snd_soc_tegra210_admaif snd_soc_tegra_pcm snd_soc_tegra210_sfc aes_ce_blk crypto_simd cryptd aes_ce_cipher snd_soc_spdif_tx ghash_ce sha2_ce sha256_arm64 snd_soc_tegra_machine_driver sha1_ce loop leds_gpio max77620_thermal snd_soc_tegra210_adsp snd_soc_tegra_utils snd_soc_simple_card_utils snd_soc_tegra210_ahub nvadsp tegra210_adma tegra_bpmp_thermal userspace_alert snd_hda_codec_hdmi snd_hda_tegra snd_hda_codec snd_hda_core spi_tegra114 binfmt_misc ina3221 nvgpu nvmap ip_tables x_tables [last unloaded: mtd]
[2024-05-15 07:09:46]  [   26.492274] CPU: 0 PID: 1454 Comm: Xorg Not tainted 5.10.104-tegra #1
[2024-05-15 07:09:46]  [   26.492278] Hardware name: Unknown NVIDIA Jetson Xavier NX Developer Kit/NVIDIA Jetson Xavier NX Developer Kit, BIOS 3.1-32827747 03/19/2023
[2024-05-15 07:09:46]  [   26.492287] pstate: 60400089 (nZCv daIf +PAN -UAO -TCO BTYPE=--)
[2024-05-15 07:09:46]  [   26.492300] pc : tegra194_cbb_err_isr+0x19c/0x1b0
[2024-05-15 07:09:46]  [   26.492307] lr : tegra194_cbb_err_isr+0x11c/0x1b0
[2024-05-15 07:09:46]  [   26.492310] sp : ffff800010003b40
[2024-05-15 07:09:46]  [   26.492314] x29: ffff800010003b40 x28: 0000000000000001 
[2024-05-15 07:09:46]  [   26.492324] x27: 0000000000000080 x26: ffffa1a009dd5200 
[2024-05-15 07:09:46]  [   26.492333] x25: ffffa1a00a74df68 x24: 0000000000000001 
[2024-05-15 07:09:46]  [   26.492345] x23: ffffa1a00a0d7000 x22: ffffa1a00a56ec50 
[2024-05-15 07:09:46]  [   26.492354] x21: 000000000000000f x20: 0000000000000005 
[2024-05-15 07:09:46]  [   26.492363] x19: ffffa1a00a56ec40 x18: 0000000000000060 
[2024-05-15 07:09:46]  [   26.492372] x17: 0000000000000000 x16: 0000000000000000 
[2024-05-15 07:09:46]  [   26.492381] x15: ffff6cd246a130f0 x14: ffffffffffffffff 
[2024-05-15 07:09:46]  [   26.492390] x13: ffffa1a00a6fdde8 x12: ffffa1a00a6fda2d 
[2024-05-15 07:09:46]  [   26.492399] x11: 0000000000000038 x10: 0101010101010101 
[2024-05-15 07:09:46]  [   26.492409] x9 : ffff800010003a50 x8 : 2a2a2a2a2a2a2a2a 
[2024-05-15 07:09:46]  [   26.492418] x7 : 2a2a2a2a2a2a2a2a x6 : 000000002b9ab7d8 
[2024-05-15 07:09:46]  [   26.492427] x5 : ffff6cd3bfdab958 x4 : 00000000fffff3a8 
[2024-05-15 07:09:46]  [   26.492436] x3 : 0000000000000001 x2 : ffffa1a00878e170 
[2024-05-15 07:09:46]  [   26.492445] x1 : ffff6cd246a12b80 x0 : 0000000100010100 
[2024-05-15 07:09:46]  [   26.492455] Call trace:
[2024-05-15 07:09:46]  [   26.492463]  tegra194_cbb_err_isr+0x19c/0x1b0
[2024-05-15 07:09:46]  [   26.492472]  __handle_irq_event_percpu+0x68/0x2a0
[2024-05-15 07:09:46]  [   26.492478]  handle_irq_event_percpu+0x40/0xa0
[2024-05-15 07:09:46]  [   26.492484]  handle_irq_event+0x50/0xf0
[2024-05-15 07:09:46]  [   26.492491]  handle_fasteoi_irq+0xc0/0x170
[2024-05-15 07:09:46]  [   26.492497]  generic_handle_irq+0x40/0x60
[2024-05-15 07:09:46]  [   26.492503]  __handle_domain_irq+0x70/0xd0
[2024-05-15 07:09:46]  [   26.492509]  efi_header_end+0xb0/0xf0
[2024-05-15 07:09:46]  [   26.492514]  el1_irq+0xd0/0x180
[2024-05-15 07:09:46]  [   26.492520]  __do_softirq+0xb4/0x3e8
[2024-05-15 07:09:46]  [   26.492527]  irq_exit+0xc0/0xe0
[2024-05-15 07:09:46]  [   26.492533]  __handle_domain_irq+0x74/0xd0
[2024-05-15 07:09:46]  [   26.492538]  efi_header_end+0xb0/0xf0
[2024-05-15 07:09:46]  [   26.492543]  el1_irq+0xd0/0x180
[2024-05-15 07:09:46]  [   26.492551]  tegra_dc_readl+0x34/0x1d0
[2024-05-15 07:09:46]  [   26.492557]  tegra_nvdisp_head_enable+0xb1c/0xf10
[2024-05-15 07:09:46]  [   26.492565]  _tegra_dc_enable+0x718/0xe10
[2024-05-15 07:09:46]  [   26.492570]  tegra_dc_enable+0x88/0x1c0
[2024-05-15 07:09:46]  [   26.492577]  tegra_fb_blank+0xf0/0x1d0
[2024-05-15 07:09:46]  [   26.492585]  fb_blank+0x64/0xb0
[2024-05-15 07:09:46]  [   26.492591]  do_fb_ioctl+0x308/0x660
[2024-05-15 07:09:46]  [   26.492597]  fb_ioctl+0x5c/0x80
[2024-05-15 07:09:46]  [   26.492606]  __arm64_sys_ioctl+0xac/0xf0
[2024-05-15 07:09:46]  [   26.492613]  el0_svc_common.constprop.0+0x80/0x1d0
[2024-05-15 07:09:46]  [   26.492619]  do_el0_svc+0x38/0xb0
[2024-05-15 07:09:46]  [   26.492628]  el0_svc+0x1c/0x30
[2024-05-15 07:09:46]  [   26.492634]  el0_sync_handler+0xa8/0xb0
[2024-05-15 07:09:46]  [   26.492639]  el0_sync+0x16c/0x180
[2024-05-15 07:09:46]  [   26.492651] Code: a9446bf9 a94573fb a8c67bfd d65f03c0 (d4210000) 
[2024-05-15 07:09:46]  [   26.492663] ---[ end trace 1e951e95146fe338 ]---
[2024-05-15 07:09:46]  [   26.507624] Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt
[2024-05-15 07:09:46]  [   26.507635] SMP: stopping secondary CPUs
[2024-05-15 07:09:46]  [   26.507691] Kernel Offset: 0x219ff85d0000 from 0xffff800010000000
[2024-05-15 07:09:46]  [   26.507695] PHYS_OFFSET: 0xffff932ec0000000
[2024-05-15 07:09:46]  [   26.507702] CPU features: 0x8240002,03802a30
[2024-05-15 07:09:46]  [   26.507705] Memory Limit: none
[2024-05-15 07:09:46]  [   26.940483] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt ]---
[2024-05-15 07:09:46]  

device tree and DEBUG UART Log:

log.txt (103.3 KB)
dts.txt (3.0 KB)

May I ask what might be causing the above?

This is fixed with a UEFI patch on rel-35.5. Please upgrade.