R3840 system failed to start

hi:

During the startup process, we encountered two instances of failure. The logs showed the same pattern each time.Please provide guidance. Thank you.

[   22.575493] CAN device driver interface
[   22.607198] EXT4-fs (nvme1n1p4): mounted filesystem d2f7dde2-3a72-4078-bbc7-6755ef5b8a70 r/w with ordered data mode. Quota mode: none.
[   22.609000] EXT4-fs (nvme1n1p2): mounted filesystem bfadb6b2-6395-44c1-8b67-5a12c25ccd5f r/w with ordered data mode. Quota mode: none.
[   22.622665] EXT4-fs (nvme1n1p1): mounted filesystem ad51fffc-e34b-4f8a-b40a-ace6d93e73da r/w with ordered data mode. Quota mode: none.
[   22.632534] mc: Linux media interface: v0.10
[   22.668743] videodev: Linux video capture interface: v2.00
[   22.697569] EXT4-fs (nvme1n1p3): mounted filesystem d07e0254-600e-43d5-8338-9ac1680cad2b r/w with ordered data mode. Quota mode: none.
[  202.924540] usbcore: registered new interface driver option
[  202.924560] usbserial: USB Serial support registered for GSM modem (1-port)
[  242.654166] INFO: task systemd-journal:489 blocked for more than 120 seconds.
[  242.654174]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654175] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654176] task:systemd-journal state:D stack:0     pid:489   tgid:489   ppid:1      flags:0x00800805
[  242.654180] Call trace:
[  242.654181]  __switch_to+0xe0/0x108
[  242.654193]  __schedule+0x36c/0x9ac
[  242.654196]  rt_mutex_schedule+0x28/0x4c
[  242.654206]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654212]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654217]  uevent_show+0x84/0x130
[  242.654228]  sysfs_kf_seq_show+0x8c/0x10c
[  242.654235]  seq_read_iter+0xe0/0x4848
[  242.654239]  kernfs_fop_read_iter+0x150/0x1bc
[  242.654241]  vfs_read+0x270/0x314
[  242.654244]  ksys_read+0x74/0x10c
[  242.654246]  __arm64_sys_read+0x1c/0x28
[  242.654248]  invoke_syscall+0x48/0x114
[  242.654253]  do_el0_svc+0x1c/0x28prop.0+0xc0/0xe0
[  242.654256]  el0_svc+0x38/0x128
[  242.654259]  el0t_64_sync_handler+0x120/0x12c
[  242.654262]  el0t_64_sync+0x194/0x198
[  242.654263] INFO: task systemd-modules:499 blocked for more than 120 seconds.
[  242.654264]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654265] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654266] task:systemd-modules state:D stack:0     pid:499   tgid:499   ppid:1      flags:0x0000000d
[  242.654268] Call trace:
[  242.654268]  __switch_to+0xe0/0x108
[  242.654270]  __schedule+0x36c/0x9ac
[  242.654272]  rt_mutex_schedule+0x28/0x4c
[  242.654274]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654276]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654283]  device_del+0x16c/0x428del+0x24/0x78
[  242.654285]  device_link_drop_managed+0xb8/0xec
[  242.654287]  device_links_driver_bound+0x1d8/0x344
[  242.654290]  driver_bound+0x74/0xc0
[  242.654295]  __driver_probe_device+0x78/0x12c
[  242.654297]  driver_probe_device+0x3c/0x164
[  242.654300]  __driver_attach+0x90/0x1a8
[  242.654303]  bus_for_each_dev+0x7c/0xdc
[  242.654305]  driver_attach+0x24/0x30
[  242.654308]  bus_add_driver+0xe4/0x208
[  242.654310]  driver_register+0x5c/0x124
[  242.654312]  __platform_driver_register+0x28/0x34
[  242.654314]  tegra_cam_rtcpu_driver_init+0x20/0x1000 [tegra_camera_rtcpu]
[  242.654334]  do_one_initcall+0x58/0x2e8
[  242.654340]  load_module+0x1ee0/0x1fbc
[  242.654343]  init_module_from_file+0x88/0xcc
[  242.654344]  __arm64_sys_finit_module+0x148/0x324
[  242.654348]  el0_svc_common.constprop.0+0x40/0xe0
[  242.654351]  do_el0_svc+0x1c/0x28
[  242.654353]  el0_svc+0x38/0x128
[  242.654356]  el0t_64_sync_handler+0x120/0x12c
[  242.654358]  el0t_64_sync+0x194/0x198
[  242.654362] INFO: task (udev-worker):595 blocked for more than 120 seconds.
[  242.654363]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654364] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654364] task:(udev-worker)   state:D stack:0     pid:595   tgid:595   ppid:590    flags:0x00000805
[  242.654366] Call trace:
[  242.654367]  __switch_to+0xe0/0x108
[  242.654369]  __schedule+0x36c/0x9ac
[  242.654371]  rt_mutex_schedule+0x28/0x4c
[  242.654373]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654375]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654378]  uevent_show+0x84/0x130
[  242.654382]  sysfs_kf_seq_show+0x8c/0x10c
[  242.654385]  seq_read_iter+0xe0/0x4848
[  242.654388]  kernfs_fop_read_iter+0x150/0x1bc
[  242.654389]  vfs_read+0x270/0x314
[  242.654391]  ksys_read+0x74/0x10c
[  242.654393]  __arm64_sys_read+0x1c/0x28
[  242.654395]  invoke_syscall+0x48/0x114
[  242.654399]  do_el0_svc+0x1c/0x28prop.0+0xc0/0xe0
[  242.654401]  el0_svc+0x38/0x128
[  242.654404]  el0t_64_sync_handler+0x120/0x12c
[  242.654406]  el0t_64_sync+0x194/0x198
[  242.654408] INFO: task (udev-worker):596 blocked for more than 120 seconds.
[  242.654409]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654410] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654412] Call trace:worker)   state:D stack:0     pid:596   tgid:596   ppid:590    flags:0x0000080d
[  242.654412]  __switch_to+0xe0/0x108
[  242.654414]  __schedule+0x36c/0x9ac
[  242.654418]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654420]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654428]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654431]  driver_probe_device+0x3c/0x164
[  242.654433]  __driver_attach+0x90/0x1a8
[  242.654436]  bus_for_each_dev+0x7c/0xdc
[  242.654438]  driver_attach+0x24/0x30
[  242.654440]  bus_add_driver+0xe4/0x208
[  242.654443]  driver_register+0x5c/0x124
[  242.654457]  do_one_initcall+0x58/0x2e8x20/0x1000 [obc_max96712]
[  242.654459]  do_init_module+0x58/0x1e8
[  242.654462]  load_module+0x1ee0/0x1fbc
[  242.654464]  init_module_from_file+0x88/0xcc
[  242.654467]  invoke_syscall+0x48/0x1140x148/0x324
[  242.654472]  do_el0_svc+0x1c/0x28prop.0+0xc0/0xe0
[  242.654474]  el0_svc+0x38/0x128
[  242.654477]  el0t_64_sync_handler+0x120/0x12c
[  242.654480]  el0t_64_sync+0x194/0x198
[  242.654482] INFO: task (udev-worker):597 blocked for more than 120 seconds.
[  242.654482]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654483] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654485] Call trace:worker)   state:D stack:0     pid:597   tgid:597   ppid:590    flags:0x0000080d
[  242.654486]  __switch_to+0xe0/0x108
[  242.654489]  __schedule+0x36c/0x9ac
[  242.654492]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654494]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654503]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654505]  driver_probe_device+0x3c/0x164
[  242.654507]  __driver_attach+0x90/0x1a8
[  242.654510]  bus_for_each_dev+0x7c/0xdc
[  242.654512]  driver_attach+0x24/0x30
[  242.654515]  bus_add_driver+0xe4/0x208
[  242.654517]  driver_register+0x5c/0x124
[  242.654519]  mc_driver_init+0x28/0x1000 [mc_t26x]
[  242.654523]  do_one_initcall+0x58/0x2e8
[  242.654527]  load_module+0x1ee0/0x1fbc
[  242.654530]  init_module_from_file+0x88/0xcc
[  242.654531]  __arm64_sys_finit_module+0x148/0x324
[  242.654535]  el0_svc_common.constprop.0+0xc0/0xe0
[  242.654537]  do_el0_svc+0x1c/0x28
[  242.654539]  el0_svc+0x38/0x128
[  242.654542]  el0t_64_sync_handler+0x120/0x12c
[  242.654544]  el0t_64_sync+0x194/0x198
[  242.654545] INFO: task (udev-worker):599 blocked for more than 120 seconds.
[  242.654546]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654547] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654548] task:(udev-worker)   state:D stack:0     pid:599   tgid:599   ppid:590    flags:0x0000080d
[  242.654549] Call trace:
[  242.654549]  __switch_to+0xe0/0x108
[  242.654551]  __schedule+0x36c/0x9ac
[  242.654553]  rt_mutex_schedule+0x28/0x4c
[  242.654555]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654556]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654564]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654567]  driver_probe_device+0x3c/0x164
[  242.654569]  __driver_attach+0x90/0x1a8
[  242.654571]  bus_for_each_dev+0x7c/0xdc
[  242.654573]  driver_attach+0x24/0x30
[  242.654576]  bus_add_driver+0xe4/0x208
[  242.654578]  driver_register+0x5c/0x124
[  242.654579]  __platform_driver_register+0x28/0x34
[  242.654580]  arm_cspmu_init+0x5c/0x1000 [arm_cspmu_module]
[  242.654585]  do_one_initcall+0x58/0x2e8
[  242.654586]  do_init_module+0x58/0x1e8
[  242.654592]  __arm64_sys_finit_module+0x148/0x324
[  242.654595]  el0_svc_common.constprop.0+0xc0/0xe0
[  242.654597]  do_el0_svc+0x1c/0x28
[  242.654600]  el0_svc+0x38/0x128
[  242.654602]  el0t_64_sync_handler+0x120/0x12c
[  242.654605]  el0t_64_sync+0x194/0x198
[  242.654606] INFO: task (udev-worker):600 blocked for more than 120 seconds.
[  242.654607]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654608] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654609] task:(udev-worker)   state:D stack:0     pid:600   tgid:600   ppid:590    flags:0x0000080d
[  242.654611] Call trace:
[  242.654612]  __switch_to+0xe0/0x108
[  242.654614]  __schedule+0x36c/0x9ac
[  242.654615]  rt_mutex_schedule+0x28/0x4c
[  242.654617]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654620]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654627]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654630]  driver_probe_device+0x3c/0x164
[  242.654632]  __driver_attach+0x90/0x1a8
[  242.654635]  bus_for_each_dev+0x7c/0xdc
[  242.654637]  driver_attach+0x24/0x30
[  242.654639]  bus_add_driver+0xe4/0x208
[  242.654641]  driver_register+0x5c/0x124
[  242.654644]  at24_init+0x48/0x1000 [at24]0
[  242.654649]  do_one_initcall+0x58/0x2e8
[  242.654653]  load_module+0x1ee0/0x1fbc
[  242.654655]  init_module_from_file+0x88/0xcc
[  242.654656]  __arm64_sys_finit_module+0x148/0x324
[  242.654659]  el0_svc_common.constprop.0+0xc0/0xe0
[  242.654662]  do_el0_svc+0x1c/0x28
[  242.654664]  el0_svc+0x38/0x128
[  242.654667]  el0t_64_sync_handler+0x120/0x12c
[  242.654669]  el0t_64_sync+0x194/0x198
[  242.654671] INFO: task (udev-worker):601 blocked for more than 120 seconds.
[  242.654672]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654673] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654674] task:(udev-worker)   state:D stack:0     pid:601   tgid:601   ppid:590    flags:0x00800805
[  242.654676] Call trace:
[  242.654676]  __switch_to+0xe0/0x108
[  242.654678]  __schedule+0x36c/0x9ac
[  242.654679]  rt_mutex_schedule+0x28/0x4c
[  242.654681]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654683]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654686]  uevent_show+0x84/0x130
[  242.654688]  sysfs_kf_seq_show+0x8c/0x10c
[  242.654692]  seq_read_iter+0xe0/0x4848
[  242.654694]  kernfs_fop_read_iter+0x150/0x1bc
[  242.654695]  vfs_read+0x270/0x314
[  242.654696]  ksys_read+0x74/0x10c
[  242.654698]  __arm64_sys_read+0x1c/0x28
[  242.654699]  invoke_syscall+0x48/0x114
[  242.654703]  do_el0_svc+0x1c/0x28prop.0+0xc0/0xe0
[  242.654706]  el0_svc+0x38/0x128
[  242.654709]  el0t_64_sync_handler+0x120/0x12c
[  242.654711]  el0t_64_sync+0x194/0x198
[  242.654714] INFO: task (udev-worker):606 blocked for more than 120 seconds.
[  242.654714]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654715] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654717] Call trace:worker)   state:D stack:0     pid:606   tgid:606   ppid:590    flags:0x0000080d
[  242.654718]  __switch_to+0xe0/0x108
[  242.654721]  __schedule+0x36c/0x9ac
[  242.654724]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654726]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654733]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654737]  driver_probe_device+0x3c/0x164
[  242.654739]  __driver_attach+0x90/0x1a8
[  242.654741]  bus_for_each_dev+0x7c/0xdc
[  242.654743]  driver_attach+0x24/0x30
[  242.654746]  bus_add_driver+0xe4/0x208
[  242.654748]  driver_register+0x5c/0x124
[  242.654751]  nvvrs_pseq_driver_init+0x20/0x1000 [nvidia_vrs_pseq]
[  242.654755]  do_one_initcall+0x58/0x2e8
[  242.654758]  load_module+0x1ee0/0x1fbc
[  242.654761]  init_module_from_file+0x88/0xcc
[  242.654762]  __arm64_sys_finit_module+0x148/0x324
[  242.654765]  el0_svc_common.constprop.0+0xc0/0xe0
[  242.654767]  do_el0_svc+0x1c/0x28
[  242.654769]  el0_svc+0x38/0x128
[  242.654772]  el0t_64_sync_handler+0x120/0x12c
[  242.654774]  el0t_64_sync+0x194/0x198
[  242.654776] INFO: task (udev-worker):609 blocked for more than 120 seconds.
[  242.654777]       Tainted: G           O       6.8.12-rt-tegra #1
[  242.654778] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.654779] task:(udev-worker)   state:D stack:0     pid:609   tgid:609   ppid:590    flags:0x0000080d
[  242.654780] Call trace:
[  242.654780]  __switch_to+0xe0/0x108
[  242.654782]  __schedule+0x36c/0x9ac
[  242.654783]  rt_mutex_schedule+0x28/0x4c
[  242.654785]  rt_mutex_slowlock_block.constprop.0+0xa8/0x178
[  242.654787]  __rt_mutex_slowlock_locked.constprop.0+0xec/0x25c
[  242.654794]  __driver_probe_device+0x78/0x12c4/0x260
[  242.654796]  driver_probe_device+0x3c/0x164
[  242.654798]  __driver_attach+0x90/0x1a8
[  242.654800]  bus_for_each_dev+0x7c/0xdc
[  242.654802]  driver_attach+0x24/0x30
[  242.654804]  bus_add_driver+0xe4/0x208
[  242.654806]  driver_register+0x5c/0x124
[  242.654807]  __platform_driver_register+0x28/0x34
[  242.654809]  tegra_tach_driver_init+0x20/0x1000 [pwm_tegra_tachometer]
[  242.654813]  do_one_initcall+0x58/0x2e8
[  242.654814]  do_init_module+0x58/0x1e8
[  242.654819]  __arm64_sys_finit_module+0x148/0x324
[  242.654822]  el0_svc_common.constprop.0+0xc0/0xe0
[  242.654824]  do_el0_svc+0x1c/0x28
[  242.654826]  el0_svc+0x38/0x128
[  242.654828]  el0t_64_sync_handler+0x120/0x12c
[  242.654830]  el0t_64_sync+0x194/0x198
[  242.654831] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
[  290.506146] systemd[1]: systemd-modules-load.service: Processes still around after SIGKILL. Ignoring.
[  291.006141] systemd[1]: systemd-journald.service: State 'stop-watchdog' timed out. Killing.
[  291.006271] systemd[1]: systemd-journald.service: Killing process 489 (systemd-journal) with signal SIGKILL.
[  380.756190] systemd[1]: systemd-modules-load.service: State 'final-sigterm' timed out. Killing.
[  380.756311] systemd[1]: systemd-modules-load.service: Killing process 499 (systemd-modules) with signal SIGKILL.
[  381.256179] systemd[1]: systemd-journald.service: Processes still around after SIGKILL. Ignoring.
[  471.006179] systemd[1]: systemd-modules-load.service: Processes still around after final SIGKILL. Entering failed mode.
[  471.006218] systemd[1]: systemd-modules-load.service: Failed with result 'timeout'.
[  471.006358] systemd[1]: systemd-modules-load.service: Unit process 499 (systemd-modules) remains running after unit stopped.
[  471.006517] systemd[1]: Failed to start systemd-modules-load.service - Load Kernel Modules.
[  471.038516] systemd[1]: Starting netfilter-persistent.service - netfilter persistent configuration...
[  471.040857] systemd[1]: Starting nv-tee-supplicant.service - OP-TEE Client Supplicant...
[  471.042295] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[  471.085410] systemd[1]: Finished netfilter-persistent.service - netfilter persistent configuration.
[  471.085557] systemd[1]: Reached target network-pre.target - Preparation for Network.
[  471.320143] systemd[1]: systemd-journald.service: State 'final-sigterm' timed out. Killing.
[  471.320180] systemd[1]: systemd-journald.service: Killing process 489 (systemd-journal) with signal SIG
[  561.256144] systemd[1]: nv-tee-supplicant.service: start-post operation timed out. Terminating.
[  561.506121] systemd[1]: systemd-journald.service: Processes still around after final SIGKILL. Entering failed mode.
[  561.506156] systemd[1]: systemd-journald.service: Failed with result 'watchdog'.
[  561.507174] systemd[1]: systemd-journald.service: Found left-over process 489 (systemd-journal) in control group while starting unit. Ignoring.
[  561.507179] systemd[1]: systemd-journald.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
[  561.588723] systemd-journald[1361]: File /run/log/journal/550b9050c105ebcc0bc24d1069df9d60/system.journal corrupted or uncleanly shut down, renaming and replacing.

开机卡死.log (113.0 KB)

log(1).txt (31.7 KB)

Is this same as this topic you posted?

hi:

yes,这个问题我们遇到了两次,没有在 developer kit 这个上面验证过

please follow above.