I encountered an RTCPU problem, the log is as follows
kern.txt (10 MB)
Mar 15 15:04:47 mxvlkica-qthdaa-3-dcu1 kernel: [ 51.396496] mv-max9296 2-0009: start streaming…
Mar 15 15:04:47 mxvlkica-qthdaa-3-dcu1 kernel: [ 51.453878] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
Mar 15 15:04:47 mxvlkica-qthdaa-3-dcu1 kernel: [ 51.454392] mv-max9296 2-0009: stop streaming…
Mar 15 15:04:50 mxvlkica-qthdaa-3-dcu1 kernel: [ 54.666989] mv-max9296 2-0010: start streaming…
Mar 15 15:04:50 mxvlkica-qthdaa-3-dcu1 kernel: [ 54.720551] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
Mar 15 15:04:50 mxvlkica-qthdaa-3-dcu1 kernel: [ 54.721063] mv-max9296 2-0010: stop streaming…
Mar 15 15:04:54 mxvlkica-qthdaa-3-dcu1 kernel: [ 57.923794] mv-max9296 2-0011: start streaming…
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.639618] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.640032] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.641484] (NULL device ): vi_capture_control_message: NULL VI channel received
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.641785] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.642210] (NULL device ): vi_capture_control_message: NULL VI channel received
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.642498] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.643204] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.643730] mv-max9296 2-0011: stop streaming…
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.908855] CPU:2, Error: rce-noc@be00000, irq=21
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909093] **************************************
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909309] CPU:2, Error:rce-noc
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909458] Error Logger : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909595] ErrLog0 : 0x80030000
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909742] Transaction Type : RD - Read, Incrementing
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.909967] Error Code : SLV
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.910120] Error Source : Target
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.910275] Error Description : Target error detected by CBB slave
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.910550] Packet header Lock : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.910701] Packet header Len1 : 3
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.910856] NOC protocol version : version >= 2.7
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.911093] ErrLog1 : 0x157600
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.911274] ErrLog2 : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.911392] RouteId : 0x157600
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.911534] InitFlow : cpu_p_i/I/0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.911702] Targflow : cbb_t/T/0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.916902] TargSubRange : 27
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.920311] SeqId : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.923197] ErrLog3 : 0x5c01814
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.926435] ErrLog4 : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.929165] Address accessed : 0x15c01814
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.933529] ErrLog5 : 0x387e31
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.936766] Master ID : RCE
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.939912] Security Group(GRPSEC): 0x3f
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.943606] [RCE] VM0 deactivating.* DATA ABORT EXCEPTION ***
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.943934] Cache : 0x1 – Bufferable
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.943956] Protection : 0x1 – Privileged, Secure, Data Access
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.948158] [RCE] R0: 15c00000 R1: 00001814 R2: 00008010 R3: 00000000
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.954349] FALCONSEC : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.954352] Virtual Queuing Channel(VQC): 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.954356] **************************************
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.954402] CPU:6, Error: cbb-noc@2300000, irq=15
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.954467] Host read timeout at address 1814
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.957517] [RCE] R4: 0bcd0188 R5: 00000000 R6: 00000000 R7: 799ce800
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.961713] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.962223] **************************************
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.962227] CPU:6, Error:cbb-noc
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.962230] Error Logger : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.962239] ErrLog0 : 0x80030000
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.962246] Transaction Type : RD - Read, Incrementing
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.967229] [RCE] R8: 0bcc27b0 R9: 0bcc27b2 R10: 0bcc27a8 R11: 00001550
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.971849] Error Code : SLV
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.971852] Error Source : Target
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.971855] Error Description : Target error detected by CBB slave
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.971868] Packet header Lock : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.976356] [RCE] R12: 0000bb80 SP: 0bcc2768 LR: 0bcaaa2c PC: 0bcaaa48
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981125] Packet header Len1 : 3
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981128] NOC protocol version : version >= 2.7
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981132] ErrLog1 : 0x9528aa
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981134] ErrLog2 : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981136] RouteId : 0x9528aa
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981139] InitFlow : rce_p2ps/I/rce_p2ps
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981141] Targflow : host1x_p2pm/T/host1x_p2pm
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981143] TargSubRange : 20
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.981148] SeqId : 0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.984464] [RCE] SPSR: 6000011f
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.987512] ErrLog3 : 0x1814
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.987518] ErrLog4 : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.991028] [RCE] DFSR: 00001008 DFAR: 15c01814 ADFSR: 00500000
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996525] Address accessed : 0x15c01814
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996528] ErrLog5 : 0x2af0fc71
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996531] Non-Modify : 0x1
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996533] AXI ID : 0x55
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996535] Master ID : RCE
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996538] Security Group(GRPSEC): 0x3f
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.996562] Cache : 0x1 – Bufferable
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 66.999785] [RCE] IFSR: 00000000 IFAR: 00000000 AIFSR: 00000000
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 67.003349] Protection : 0x3 – Privileged, Non-Secure, Data Access
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 67.003356] FALCONSEC : 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 67.088840] Virtual Queuing Channel(VQC): 0x0
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 67.093475] **************************************
Mar 15 15:05:03 mxvlkica-qthdaa-3-dcu1 kernel: [ 67.099229] mttcan c310000.mttcan can0: mttcan_poll_ir: some msgs lost on in Q0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 68.927644] bc00000.rtcpu:hsp-vm1: request 0x43000000: response timeout
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 68.927954] bc00000.rtcpu:hsp-vm1: PM_SUSPEND failed: 0xffffff92
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 68.927960] tegra186-cam-rtcpu bc00000.rtcpu: RTCPU suspend failed, resetting it
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730038] CPU:0, Error: rce-noc@be00000, irq=20
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730240] **************************************
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730438] CPU:0, Error:rce-noc
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730587] Error Logger : 1
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730782] ErrLog0 : 0x801f0602
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.730933] Transaction Type : RDW - Read, Wrap
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731131] Error Code : TMO
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731293] Error Source : Target NIU
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731460] Error Description : Target time-out error
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731663] Packet header Lock : 0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731807] Packet header Len1 : 31
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.731954] NOC protocol version : version >= 2.7
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732153] ErrLog1 : 0xcc006
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732295] ErrLog2 : 0x0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732416] RouteId : 0xcc006
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732546] InitFlow : cpu_m_i/I/0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732692] Targflow : ast0_t/T/0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.732866] TargSubRange : 0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.733116] SeqId : 0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.733636] ErrLog3 : 0x47000
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.734145] ErrLog4 : 0x0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.734613] Address accessed : 0x40047000
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.735310] ErrLog5 : 0x387e3f
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.735838] Master ID : RCE
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.736331] Security Group(GRPSEC): 0x3f
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.736981] Cache : 0xf – Bufferable Modifiable Read-Allocate Write-Allocate
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.740165] Protection : 0xf – Privileged, Non-Secure, Instruction Access
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.747340] FALCONSEC : 0x0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.750315] Virtual Queuing Channel(VQC): 0x0
Mar 15 15:05:05 mxvlkica-qthdaa-3-dcu1 kernel: [ 69.755060] **************************************
Mar 15 15:05:07 mxvlkica-qthdaa-3-dcu1 kernel: [ 70.943629] bc00000.rtcpu:hsp-vm1: request 0x40636b81: response timeout
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.959626] bc00000.rtcpu:hsp-vm1: request 0x4024b144: response timeout
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.959945] tegra186-cam-rtcpu bc00000.rtcpu: rce full reset, retry 2/3
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.960671] CPU:0, Error: rce-noc@be00000, irq=20
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.960862] **************************************
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961058] CPU:0, Error:rce-noc
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961195] Error Logger : 1
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961366] ErrLog0 : 0x801f0600
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961508] Transaction Type : RD - Read, Incrementing
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961718] Error Code : TMO
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.961856] Error Source : Target NIU
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962043] Error Description : Target time-out error
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962252] Packet header Lock : 0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962393] Packet header Len1 : 31
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962536] NOC protocol version : version >= 2.7
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962727] ErrLog1 : 0xcc002
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962862] ErrLog2 : 0x0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.962973] RouteId : 0xcc002
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.963221] InitFlow : cpu_m_i/I/0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.963839] Targflow : ast0_t/T/0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.964414] TargSubRange : 0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.964944] SeqId : 0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.965395] ErrLog3 : 0x30000060
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.965968] ErrLog4 : 0x0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.966424] Address accessed : 0x70000060
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.967116] ErrLog5 : 0x387e73
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.969578] Master ID : RCE
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.972728] Security Group(GRPSEC): 0x3f
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.976753] Cache : 0x3 – Bufferable Modifiable
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.981828] Protection : 0x3 – Privileged, Non-Secure, Data Access
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.988231] FALCONSEC : 0x0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.991628] Virtual Queuing Channel(VQC): 0x0
Mar 15 15:05:09 mxvlkica-qthdaa-3-dcu1 kernel: [ 72.996196] **************************************
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.975591] bc00000.rtcpu:hsp-vm1: request 0x40e651a4: response timeout
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.975883] tegra186-cam-rtcpu bc00000.rtcpu: rce full reset, retry 3/3
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.976514] CPU:4, Error: rce-noc@be00000, irq=20
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.976726] **************************************
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.976941] CPU:4, Error:rce-noc
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.977090] Error Logger : 1
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.977275] ErrLog0 : 0x801f0600
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.977430] Transaction Type : RD - Read, Incrementing
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.977690] Error Code : TMO
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.977836] Error Source : Target NIU
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978011] Error Description : Target time-out error
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978236] Packet header Lock : 0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978402] Packet header Len1 : 31
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978569] NOC protocol version : version >= 2.7
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978783] ErrLog1 : 0xcc002
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.978923] ErrLog2 : 0x0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.979040] RouteId : 0xcc002
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.979178] InitFlow : cpu_m_i/I/0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.979790] Targflow : ast0_t/T/0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.980361] TargSubRange : 0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.980895] SeqId : 0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.981343] ErrLog3 : 0x30000060
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.981910] ErrLog4 : 0x0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.982355] Address accessed : 0x70000060
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.983053] ErrLog5 : 0x387e73
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.985528] Master ID : RCE
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.988679] Security Group(GRPSEC): 0x3f
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.992703] Cache : 0x3 – Bufferable Modifiable
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 74.997779] Protection : 0x3 – Privileged, Non-Secure, Data Access
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 75.004429] FALCONSEC : 0x0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 75.007579] Virtual Queuing Channel(VQC): 0x0
Mar 15 15:05:11 mxvlkica-qthdaa-3-dcu1 kernel: [ 75.012150] **************************************
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.995648] bc00000.rtcpu:hsp-vm1: request 0x40a79117: response timeout
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996262] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996270] tegra186-cam-rtcpu bc00000.rtcpu: cannot reboot while suspended
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996305] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996310] tegra186-cam-rtcpu bc00000.rtcpu: cannot reboot while suspended
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996332] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996336] tegra186-cam-rtcpu bc00000.rtcpu: cannot reboot while suspended
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996359] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996363] tegra186-cam-rtcpu bc00000.rtcpu: cannot reboot while suspended
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996382] tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
Mar 15 15:05:13 mxvlkica-qthdaa-3-dcu1 kernel: [ 76.996388] tegra186-cam-rtcpu bc00000.rtcpu: cannot reboot while suspended
1: What I want to confirm is whether the following error message will cause an error in the RTCPU
Mar 15 15:04:54 mxvlkica-qthdaa-3-dcu1 kernel: [ 57.923794] mv-max9296 2-0011: start streaming…
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.639618] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.640032] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.641484] (NULL device *): vi_capture_control_message: NULL VI channel received
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.641785] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.642210] (NULL device *): vi_capture_control_message: NULL VI channel received
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.642498] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.643204] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
Mar 15 15:04:56 mxvlkica-qthdaa-3-dcu1 kernel: [ 60.643730] mv-max9296 2-0011: stop streaming…
2: Because there are too many error logs on the RTCPU causing too many system logs, I want to turn off the printing below
tegra186-cam-rtcpu bc00000.rtcpu: Alert: Camera RTCPU gone bad! restoring it immediately!!
From the code logic point of view, I shield the corresponding interrupt, modify the following will I bring other questions
diff --git a/drivers/platform/tegra/rtcpu/rtcpu-monitor.c b/drivers/platform/tegra/rtcpu/rtcpu-monitor.c
index 214e4a68f…93b8ef3f1 100644
— a/drivers/platform/tegra/rtcpu/rtcpu-monitor.c
+++ b/drivers/platform/tegra/rtcpu/rtcpu-monitor.c
@@ -52,7 +52,7 @@ static void tegra_camrtc_mon_wdt_worker(struct work_struct *work)
tegra_camrtc_mon_restore_rtcpu(cam_rtcpu_mon);
/* Enable WDT IRQ */
-
enable_irq(cam_rtcpu_mon->wdt_irq);
-
//enable_irq(cam_rtcpu_mon->wdt_irq);
}
I use the xavier platform, jetpack5.1.1
The problem could be capture failed from the sensor cause RTCPU go to unknow state.
I would suggest checking the stability for the sensor capture.
Thanks
We have ensured that the 8 cameras are normal, but there are still RTCPU errors. Do you have any suggestions，Log as follows
kern.log (1.7 MB)
Mar 22 15:19:45 mxvlkica-qthdaa-251-dcu1 kernel: [ 77.255033] mv-max9296 2-0004: start streaming…
Mar 22 15:19:45 mxvlkica-qthdaa-251-dcu1 kernel: [ 77.313999] mv-max9296 2-0004: stop streaming…
Mar 22 15:19:49 mxvlkica-qthdaa-251-dcu1 kernel: [ 81.573736] mv-max9296 2-0005: start streaming…
Mar 22 15:19:49 mxvlkica-qthdaa-251-dcu1 kernel: [ 81.613977] mv-max9296 2-0005: stop streaming…
Mar 22 15:19:53 mxvlkica-qthdaa-251-dcu1 kernel: [ 85.842722] mv-max9296 2-0006: start streaming…
Mar 22 15:19:53 mxvlkica-qthdaa-251-dcu1 kernel: [ 85.880633] mv-max9296 2-0006: stop streaming…
Mar 22 15:19:58 mxvlkica-qthdaa-251-dcu1 kernel: [ 90.105690] mv-max9296 2-0007: start streaming…
Mar 22 15:19:58 mxvlkica-qthdaa-251-dcu1 kernel: [ 90.147299] mv-max9296 2-0007: stop streaming…
Mar 22 15:20:02 mxvlkica-qthdaa-251-dcu1 kernel: [ 94.347236] mv-max9296 2-0008: start streaming…
Mar 22 15:20:02 mxvlkica-qthdaa-251-dcu1 kernel: [ 94.380569] mv-max9296 2-0008: stop streaming…
Mar 22 15:20:06 mxvlkica-qthdaa-251-dcu1 kernel: [ 98.601893] mv-max9296 2-0009: start streaming…
Mar 22 15:20:06 mxvlkica-qthdaa-251-dcu1 kernel: [ 98.647291] mv-max9296 2-0009: stop streaming…
Mar 22 15:20:10 mxvlkica-qthdaa-251-dcu1 kernel: [ 102.866760] mv-max9296 2-0010: start streaming…
Mar 22 15:20:10 mxvlkica-qthdaa-251-dcu1 kernel: [ 102.913946] mv-max9296 2-0010: stop streaming…
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176161] CPU:4, Error: cbb-noc@2300000, irq=15
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176274] Host read timeout at address 1814
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176379] **************************************
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176381] CPU:4, Error:cbb-noc
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176384] Error Logger : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.176394] ErrLog0 : 0x80030000
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.177184] Transaction Type : RD - Read, Incrementing
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.177420] Error Code : SLV
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.177567] Error Source : Target
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.177720] Error Description : Target error detected by CBB slave
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.177999] Packet header Lock : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178152] Packet header Len1 : 3
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178308] NOC protocol version : version >= 2.7
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178551] ErrLog1 : 0x9528aa
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178729] ErrLog2 : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178846] RouteId : 0x9528aa
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.178986] InitFlow : rce_p2ps/I/rce_p2ps
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.185583] Targflow : host1x_p2pm/T/host1x_p2pm
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.190570] TargSubRange : 20
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.193895] SeqId : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.196607] ErrLog3 : 0x1814
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.199583] ErrLog4 : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.202474] Address accessed : 0x15c01814
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.203963] [RCE] VM0 deactivating.VM0 activating.VM0 deactivating.*** DATA ABORT EXCEPTION ***
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.206670] ErrLog5 : 0x2af0fc71
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.206678] Non-Modify : 0x1
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.210371] [RCE] R0: 15c00000 R1: 00001814 R2: 00008010 R3: 00000000
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213757] AXI ID : 0x55
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213761] Master ID : RCE
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213763] Security Group(GRPSEC): 0x3f
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213766] Cache : 0x1 – Bufferable
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213769] Protection : 0x3 – Privileged, Non-Secure, Data Access
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213771] FALCONSEC : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213775] Virtual Queuing Channel(VQC): 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213778] **************************************
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.213801] CPU:0, Error: rce-noc@be00000, irq=21
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.216955] [RCE] R4: 0bcd0188 R5: 00000000 R6: 00000000 R7: 799ce800
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219883] **************************************
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219885] CPU:0, Error:rce-noc
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219889] Error Logger : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219900] ErrLog0 : 0x80030000
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219902] Transaction Type : RD - Read, Incrementing
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219905] Error Code : SLV
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219907] Error Source : Target
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219909] Error Description : Target error detected by CBB slave
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219918] Packet header Lock : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219920] Packet header Len1 : 3
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219922] NOC protocol version : version >= 2.7
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219924] ErrLog1 : 0x157600
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219942] ErrLog2 : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.219954] RouteId : 0x157600
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.224131] [RCE] R8: 0bcc27b0 R9: 0bcc27b2 R10: 0bcc27a8 R11: 000007f0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228195] InitFlow : cpu_p_i/I/0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228198] Targflow : cbb_t/T/0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228201] TargSubRange : 27
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228203] SeqId : 0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228205] ErrLog3 : 0x5c01814
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228207] ErrLog4 : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228211] Address accessed : 0x15c01814
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228228] ErrLog5 : 0x387e31
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228231] Master ID : RCE
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228233] Security Group(GRPSEC): 0x3f
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228236] Cache : 0x1 – Bufferable
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228245] Protection : 0x1 – Privileged, Secure, Data Access
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228249] FALCONSEC : 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.228262] Virtual Queuing Channel(VQC): 0x0
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.235048] [RCE] R12: 0000a8e6 SP: 0bcc2768 LR: 0bcaaa2c PC: 0bcaaa48
Mar 22 15:20:17 mxvlkica-qthdaa-251-dcu1 kernel: [ 109.238172] **************************************
It looks like mxvlkica-qthdaa-3-dcu access to NULL point cause the problem. Doesn’t looks like RTCPU’s firmware problem.
However, the latest kernel log does not find any errors in accessing null Pointers，Could you point out the problem in that place？
Please take a look at the last log. The opening and closing of the seven cameras are normal, but there will be a problem about 6 seconds after the last camera is closed, and there are no other error messages. We cannot judge what went wrong？
Please apply below patch to try.
0001-camera-vi5-fix-stream-on-off-memory-leakage.patch (6.4 KB)
0001-capture-ivc-fix-multi-cam-race-condition.patch (2.4 KB)
0001-vi5-continue-captures-even-after-corr-errors.patch (2.3 KB)
0001-vi5_fops-fix-mem-leak.patch (2.9 KB)
I compiled the Image by combining four patches, but the kernel crashed immediately after starting the camera. 0001-capture-ivc-fix-multi-cam-race-condition.patch and 0001-vi5_fops-fix-mem-leak.patch we have combined before, this one is for memory leaks. The feeling did not solve our problem. From the log we appeared to ask, the feeling was that the RTCPU was abnormal due to the camera shutdown
