Mmc0 error cause system crash

Module: Jetson AGX Orin 32G

software: Jetpack-5.1.4

Sometimes We met a mmc0 error in reboot test 。

If we don’t want to upgrade to the jetpack6.2 ,How can we solve this problem?

error log:

[ 9989.191774] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[ 9989.201661] —[ end trace 24335a774da0814a ]—
[10021.853583] mmc0: Timeout waiting for hardware interrupt.
[10021.853759] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[10021.853960] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000505
[10021.854159] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000000
[10021.854361] mmc0: sdhci: Argument: 0x00010000 | Trn mode: 0x00000023
[10021.854550] mmc0: sdhci: Present: 0x01fb00f0 | Host ctl: 0x00000039
[10021.854749] mmc0: sdhci: Power: 0x00000001 | Blk gap: 0x00000000
[10021.854962] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000000f
[10021.855157] mmc0: sdhci: Timeout: 0x0000000e | Int stat: 0x00000000
[10021.855359] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00fc0003
[10021.855566] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[10021.855752] mmc0: sdhci: Caps: 0x3f6cd08c | Caps_1: 0x18002f73
[10021.856368] mmc0: sdhci: Cmd: 0x00000d1a | Max curr: 0x00000000
[10021.857335] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0x0400658d
[10021.858300] mmc0: sdhci: Resp[2]: 0x0fc30000 | Resp[3]: 0x00000240
[10021.859262] mmc0: sdhci: Host ctl2: 0x0000300d
[10021.862711] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x0000007fffffe210
[10021.869987] mmc0: sdhci: ============================================
[10021.876707] mmc0: cache flush error -110
[10083.293869] mmc0: cqhci: timeout for tag 0
[10083.294011] mmc0: cqhci: ============ CQHCI REGISTER DUMP ===========
[10083.294234] mmc0: cqhci: Caps: 0x000020c8 | Version: 0x00000510
[10083.294457] mmc0: cqhci: Config: 0x00001101 | Control: 0x00000100
[10083.294658] mmc0: cqhci: Int stat: 0x00000000 | Int enab: 0x00000006
[10083.294876] mmc0: cqhci: Int sig: 0x00000006 | Int Coal: 0x00000000
[10083.295090] mmc0: cqhci: TDL base: 0xffffb000 | TDL up32: 0x0000007f
[10083.295298] mmc0: cqhci: Doorbell: 0x7fffffff | TCN: 0x00000000
[10083.295499] mmc0: cqhci: Dev queue: 0x00000000 | Dev Pend: 0x00000000
[10083.295698] mmc0: cqhci: Task clr: 0x00000000 | SSC1: 0x00001000
[10083.295897] mmc0: cqhci: SSC2: 0x00000001 | DCMD rsp: 0x00000800
[10083.296092] mmc0: cqhci: RED mask: 0xfdf9a080 | TERRI: 0x062e050d
[10083.296460] mmc0: cqhci: Resp idx: 0x0000002e | Resp arg: 0x00000900
[10083.297423] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[10083.298416] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000505
[10083.299356] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000000
[10083.303810] mmc0: sdhci: Argument: 0x00010000 | Trn mode: 0x00000023
[10083.310279] mmc0: sdhci: Present: 0x01fb00f0 | Host ctl: 0x00000038
[10083.316753] mmc0: sdhci: Power: 0x00000001 | Blk gap: 0x00000000
[10083.323238] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000000f
[10083.329710] mmc0: sdhci: Timeout: 0x0000000e | Int stat: 0x00000000
[10083.336177] mmc0: sdhci: Int enab: 0x02ff4000 | Sig enab: 0x02fc4000
[10083.342653] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[10083.349124] mmc0: sdhci: Caps: 0x3f6cd08c | Caps_1: 0x18002f73
[10083.355599] mmc0: sdhci: Cmd: 0x00000d1a | Max curr: 0x00000000
[10083.362074] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0x0400658d
[10083.368544] mmc0: sdhci: Resp[2]: 0x0fc30000 | Resp[3]: 0x00000240
[10083.375018] mmc0: sdhci: Host ctl2: 0x0000300d
[10083.379394] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x0000007fffffe210
[10083.386669] mmc0: sdhci: ============================================
[10083.393140] mmc0: running CQE recovery
[10083.901851] mmc0: cqhci: Failed to halt
[10083.902338] CPU:0, Error: cbb-fabric@0x13a00000, irq=34
[10083.902513] **************************************
[10083.902668] CPU:0, Error:cbb-fabric, Errmon:2
[10083.902799] Error Code : SLAVE_ERR
[10083.902908] Overflow : Multiple SLAVE_ERR
[10083.903045]
[10083.903093] Error Code : SLAVE_ERR
[10083.903206] MASTER_ID : CCPLEX
[10083.903302] Address : 0x3460008
[10083.903413] Cache : 0x1 – Bufferable
[10083.903536] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[10083.903729] Access_Type : Write
[10083.903836] Access_ID : 0x7
[10083.903842] Fabric : cbb-fabric
[10083.904038] Slave_Id : 0x3a
[10083.904129] Burst_length : 0x0
[10083.904222] Burst_type : 0x1
[10083.904321] Beat_size : 0x2
[10083.904410] VQC : 0x0
[10083.904537] GRPSEC : 0x7e
[10083.905001] FALCONSEC : 0x0
[10083.905466] **************************************
[10083.906229] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[10083.907940] —[ end trace 24335a774da0814b ]—
[10096.093925] mmc0: Timeout waiting for hardware interrupt.
[10096.094106] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[10096.094320] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000505
[10096.094514] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000000
[10096.094721] mmc0: sdhci: Argument: 0x00010000 | Trn mode: 0x00000023
[10096.094919] mmc0: sdhci: Present: 0x01fb00f0 | Host ctl: 0x00000039
[10096.095115] mmc0: sdhci: Power: 0x00000001 | Blk gap: 0x00000000
[10096.095315] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000000f
[10096.095504] mmc0: sdhci: Timeout: 0x0000000e | Int stat: 0x00000000
[10096.095690] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00fc0003
[10096.095877] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[10096.096065] mmc0: sdhci: Caps: 0x3f6cd08c | Caps_1: 0x18002f73
[10096.096718] mmc0: sdhci: Cmd: 0x00000d1a | Max curr: 0x00000000
[10096.097683] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0x0400658d
[10096.098649] mmc0: sdhci: Resp[2]: 0x0fc30000 | Resp[3]: 0x00000240
[10096.099615] mmc0: sdhci: Host ctl2: 0x0000300d
[10096.103062] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x0000007fffffe210
[10096.110342] mmc0: sdhci: ============================================
[10096.117262] CPU:0, Error: cbb-fabric@0x13a00000, irq=34
[10096.121961] **************************************
[10096.126861] CPU:0, Error:cbb-fabric, Errmon:2
[10096.131328] Error Code : SLAVE_ERR
[10096.135170] Overflow : Multiple SLAVE_ERR
[10096.139551]
[10096.141122] Error Code : SLAVE_ERR
[10096.145058] MASTER_ID : CCPLEX
[10096.148383] Address : 0x3460008
[10096.151975] Cache : 0x1 – Bufferable
[10096.156173] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[10096.162910] Access_Type : Write
[10096.166497] Access_ID : 0x7
[10096.166503] Fabric : cbb-fabric
[10096.173321] Slave_Id : 0x3a
[10096.176299] Burst_length : 0x0
[10096.179708] Burst_type : 0x1
[10096.183033] Beat_size : 0x2
[10096.186182] VQC : 0x0
[10096.189070] GRPSEC : 0x7e
[10096.192220] FALCONSEC : 0x0
[10096.195198] **************************************
[10096.200123] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[10096.209989] —[ end trace 24335a774da0814c ]—
[10108.125823] mmc0: Timeout waiting for hardware interrupt.
[10108.126005] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[10108.126208] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000505
[10108.126408] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000000
[10108.126609] mmc0: sdhci: Argument: 0x00010000 | Trn mode: 0x00000023
[10108.126809] mmc0: sdhci: Present: 0x01fb00f0 | Host ctl: 0x00000039
[10108.127005] mmc0: sdhci: Power: 0x00000001 | Blk gap: 0x00000000
[10108.127209] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000000f
[10108.127407] mmc0: sdhci: Timeout: 0x0000000e | Int stat: 0x00000000
[10108.127594] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00fc0003
[10108.127786] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[10108.127969] mmc0: sdhci: Caps: 0x3f6cd08c | Caps_1: 0x18002f73
[10108.128617] mmc0: sdhci: Cmd: 0x00000d1a | Max curr: 0x00000000
[10108.129580] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0x0400658d
[10108.130548] mmc0: sdhci: Resp[2]: 0x0fc30000 | Resp[3]: 0x00000240
[10108.131516] mmc0: sdhci: Host ctl2: 0x0000300d
[10108.134962] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x0000007fffffe210
[10108.142241] mmc0: sdhci: ============================================
[10108.148811] CPU:0, Error: cbb-fabric@0x13a00000, irq=34
[10108.153860] **************************************
[10108.158757] CPU:0, Error:cbb-fabric, Errmon:2
[10108.163223] Error Code : SLAVE_ERR
[10108.167068] Overflow : Multiple SLAVE_ERR
[10108.171449]
[10108.173018] Error Code : SLAVE_ERR
[10108.176958] MASTER_ID : CCPLEX
[10108.180282] Address : 0x3460008
[10108.183869] Cache : 0x1 – Bufferable
[10108.188070] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[10108.194806] Access_Type : Write
[10108.198394] Access_ID : 0x7
[10108.198397] Fabric : cbb-fabric
[10108.205218] Slave_Id : 0x3a
[10108.208196] Burst_length : 0x0
[10108.211607] Burst_type : 0x1
[10108.214931] Beat_size : 0x2
[10108.218082] VQC : 0x0
[10108.220969] GRPSEC : 0x7e
[10108.224120] FALCONSEC : 0x0
[10108.227095] **************************************
[10108.232020] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[10108.241903] —[ end trace 24335a774da0814d ]—
[10120.157821] mmc0: Timeout waiting for hardware interrupt.
[10120.158017] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[10120.158223] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000505
[10120.158419] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000000
[10120.158618] mmc0: sdhci: Argument: 0x00010000 | Trn mode: 0x00000023
[10120.158813] mmc0: sdhci: Present: 0x01fb00f0 | Host ctl: 0x00000039
[10120.159015] mmc0: sdhci: Power: 0x00000001 | Blk gap: 0x00000000
[10120.159219] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000000f
[10120.159400] mmc0: sdhci: Timeout: 0x0000000e | Int stat: 0x00000000
[10120.159588] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00fc0003
[10120.159778] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[10120.159966] mmc0: sdhci: Caps: 0x3f6cd08c | Caps_1: 0x18002f73
[10120.160614] mmc0: sdhci: Cmd: 0x00000d1a | Max curr: 0x00000000
[10120.161577] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0x0400658d
[10120.162547] mmc0: sdhci: Resp[2]: 0x0fc30000 | Resp[3]: 0x00000240
[10120.163514] mmc0: sdhci: Host ctl2: 0x0000300d
[10120.166951] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x0000007fffffe210
[10120.174231] mmc0: sdhci: ============================================
[10120.180956] CPU:0, Error: cbb-fabric@0x13a00000, irq=34
[10120.185853] **************************************
[10120.190751] CPU:0, Error:cbb-fabric, Errmon:2
[10120.195217] Error Code : SLAVE_ERR
[10120.199064] Overflow : Multiple SLAVE_ERR
[10120.203443]
[10120.205012] Error Code : SLAVE_ERR
[10120.208949] MASTER_ID : CCPLEX
[10120.212274] Address : 0x3460008
[10120.215863] Cache : 0x1 – Bufferable
[10120.220064] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[10120.226799] Access_Type : Write
[10120.230389] Access_ID : 0x7
[10120.230395] Fabric : cbb-fabric
[10120.237212] Slave_Id : 0x3a
[10120.240188] Burst_length : 0x0
[10120.243599] Burst_type : 0x1
[10120.246925] Beat_size : 0x2
[10120.250073] VQC : 0x0
[10120.252964] GRPSEC : 0x7e
[10120.256111] FALCONSEC : 0x0
[10120.259087] **************************************
[10120.264011] WARNING: CPU: 0 PID: 0 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[10120.273889] —[ end trace 24335a774da0814e ]—

please move to rel-36.4.3 and test if this issue would really not happen on devkit first.

Just for debug purpose.

This issue has been tested on r3644 and no problems were found. Currently, the project is in a critical rush and there is no time to replace r3644. Please help solve this problem on r3560.

May I ask if there has been any progress?