Kernl Panic:CPU:0, Error:rce-fabric, Errmon:2 Error Code : FIREWALL_ERR

Hi NV,

When we did stress testing, a kernel panic error occurred.

==Loop operation :
1,Using V4L2 API Open camera video stream(ioctl(camera_ctx.fd, VIDIOC_DQBUF, &bufferinfo),
and displayed via OpenCV API
2,reboot system

UART log:
2xisx031+4xLC016_minicom.txt (60.2 MB)

Please help with analysis, thank you!

hello future.wang,

according to your description, it seems you’re using v4l2 API to fetch the camera stream.
since you’re working with Orin series, which has OEM firewall.
please check whether you’ve access to registers directly.

anyways, please see-also Topic 234321 for the approaches to disable firewall config.

Hi Jerry,

tegra234-mb2-bct-scr-p3701-0000-override.dts
Topic 234321 The modified content is as follows:

/ {
    tfc {
        reg@401 { /* GPIO_G_SCR_07_0 */
            exclusion-info = <2>;
            value = <0x18000606>;
        };

        reg@419 { /* GPIO_CTL4_SCR_0 */
            exclusion-info = <2>;
            value = <0x18000606>;
        };

        reg@5114 { /* CBB_CENTRAL_CBB_FIREWALL_PWM5_BLF, READ_CTL */
            exclusion-info = <2>;
            value = <0x0010000a>;
        };

        reg@5115 { /* CBB_CENTRAL_CBB_FIREWALL_PWM5_BLF, WRITE_CTL */
            exclusion-info = <2>;
            value = <0x0010000a>;
        };

        reg@5402 { /* CBB_CENTRAL_CBB_FIREWALL_SPI3_BLF, READ_CTL */
            exclusion-info = <2>;
            value = <0x0f000042>;
        };

        reg@5403 { /* CBB_CENTRAL_CBB_FIREWALL_SPI3_BLF, WRITE_CTL */
            exclusion-info = <2>;
            value = <0x0f000042>;
        };

        reg@4673 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, READ_CTL */
            exclusion-info = <0>;
        };                                                                                                                                                                                                                                                                         

        reg@4674 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, WRITE_CTL */
            exclusion-info = <0>;
        };

        reg@4675 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, CTL_SETTING */
            exclusion-info = <0>;
        };
    };  
};

and
tegra234-firewall-config-base.dtsi

        reg@4673 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, READ_CTL */
            exclusion-info = <0>;
            value = <0x00004000>;
        };

        reg@4674 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, WRITE_CTL */
            exclusion-info = <0>;
            value = <0x00004000>;
        };

        reg@4675 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, CTL_SETTING */                                                                                                                                                                                                    
            exclusion-info = <0>;
            value = <0x80030000>;
        };


I modified the two files above at the same time,then re-flash the image
BUT the error still exists, the log content is exactly the same.
0529_minicom.txt (1.4 MB)

hello future.wang,

it looks the same failure,
please share your flash command-line for reference as well.

Hi Jerry

I meet the same problem when try to access NVCSI register for debug purpose using devmem command. Jetpack 5.1.3 is used on AGX orin devkit. Could you please give some help for me to access NVCSI registers using busybox devmem tool ?

[ 272.121450] CPU:0, Error: cbb-fabric@0x13a00000, irq=34
ssa@ubuntu:~$ [ 272.126865] **************************************
[ 272.131789] CPU:0, Error:cbb-fabric, Errmon:2
[ 272.136275] Error Code : SLAVE_ERR
[ 272.140141]
[ 272.141668] Error Code : SLAVE_ERR
[ 272.145527] MASTER_ID : CCPLEX
[ 272.149023] Address : 0x15a11ce8
[ 272.152702] Cache : 0x0 – Device Non-Bufferable
[ 272.157896] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[ 272.164875] Access_Type : Read
[ 272.168373] Access_ID : 0x13
[ 272.168376] Fabric : cbb-fabric
[ 272.175286] Slave_Id : 0x3
[ 272.178428] Burst_length : 0x0
[ 272.181924] Burst_type : 0x1
[ 272.185249] Beat_size : 0x2
[ 272.188479] VQC : 0x0
[ 272.191259] GRPSEC : 0x7e
[ 272.194315] FALCONSEC : 0x0
[ 272.197545] **************************************
[ 272.202616] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[ 272.212932] —[ end trace 362d9e6a12e80d49 ]—
[ 272.217760] Host read timeout at address 15a11ce8

hello zxgdut,

no, you’re having different error logs…

such SLAVE_ERR may due to communication failure.
did you have putting camera streaming on to access those NVCSI registers.

1 Like

No, no camera stream is on when I use devmem. I try to put camera streaming on, the NVCSI registers can be accessed now. Thank you so much!

Hi Jerry,

flash cmd:
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only

log:
flash.txt (49.4 KB)

Thanks!

hello future.wang,

since you’re working with Orin NX.
it’s using tegra234-mb2-bct-scr-p3767-0000.dts as SCR config, and tegra234-mb2-bct-scr-p3701-0000-override.dts as override files.
so, you should only revise the override file for adding CBB_FIREWALL settings.

as you may know… it’s initrd to generate the image locally and then flashing the target.
you should re-generate the image again for flashing the target.

Hi Jerry,

tegra234-mb2-bct-scr-p3701-0000-override.dts 
The modified version is as follows:

       reg@5114 { /* CBB_CENTRAL_CBB_FIREWALL_PWM5_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@5115 { /* CBB_CENTRAL_CBB_FIREWALL_PWM5_BLF, WRITE_CTL */            exclusion-info = <0>;        };
        reg@5402 { /* CBB_CENTRAL_CBB_FIREWALL_SPI3_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@5403 { /* CBB_CENTRAL_CBB_FIREWALL_SPI3_BLF, WRITE_CTL */            exclusion-info = <0>;        };        
        reg@5138 { /* CBB_CENTRAL_CBB_FIREWALL_QSPI0_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@5139 { /* CBB_CENTRAL_CBB_FIREWALL_QSPI0_BLF, WRITE_CTL */            exclusion-info = <0>;        };        
        reg@3224 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@3225 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, WRITE_CTL */            exclusion-info = <0>;        };
        reg@3226 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, CTL_SETTING */            exclusion-info = <0>;        };        
        reg@3362 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@3363 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, WRITE_CTL */            exclusion-info = <0>;        };
        reg@3364 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, CTL_SETTING */            exclusion-info = <0>;        };       
        reg@3443 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, READ_CTL */            exclusion-info = <0>;        };
        reg@3444 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, WRITE_CTL */            exclusion-info = <0>;        };
        reg@3445 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, CTL_SETTING */            exclusion-info = <0>;        }; 
tegra234-mb2-bct-scr-p3767-0000.dts
The modified version is as follows:

reg@5138 { /* CBB_CENTRAL_CBB_FIREWALL_QSPI0_BLF, READ_CTL */           exclusion-info = <0>;         };
reg@5139 { /* CBB_CENTRAL_CBB_FIREWALL_QSPI0_BLF, WRITE_CTL */          exclusion-info = <0>;         };
reg@3224 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, READ_CTL */         exclusion-info = <0>;          };
reg@3225 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, WRITE_CTL */         exclusion-info = <0>;          };
reg@3226 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_MN_FSI_BLF, CTL_SETTING */           exclusion-info = <0>;        };
reg@3362 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, READ_CTL */           exclusion-info = <0>;     };
reg@3363 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, WRITE_CTL */          exclusion-info = <0>;       };
reg@3364 { /* CBB_CENTRAL_CBB_FIREWALL_CBB_SN_FSI_SLAVE_BLF, CTL_SETTING */         exclusion-info = <0>;       };
reg@3443 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, READ_CTL */                exclusion-info = <0>;        };
reg@3444 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, WRITE_CTL */              exclusion-info = <0>;        };
reg@3445 { /* CBB_CENTRAL_CBB_FIREWALL_CSITE_FSI_BLF, CTL_SETTING */          exclusion-info = <0>;          };

Are the modifications to the above two files correct?
I also have a question. Since these two dts files are in the bootloader directory,
they will not be compiled into dtb files, right? Why do we need to re-generate the image?

hello future.wang,

what’s your actual use-case?
you should only updating below three if you would like to configure OEM firewall to access NVCSI registers,
for instance,
reg@4673 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, READ_CTL */
reg@4674 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, WRITE_CTL */
reg@4675 { /* CBB_CENTRAL_CBB_FIREWALL_NVCSI_ENGINE_BLF, CTL_SETTING */

since you’re working with initrd script.
it’s the image under… $OUT/Linux_for_Tegra/tools/kernel_flash/images/ that flash to your target actually.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.