Open camera occur system reboot on jetpack 4.4

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

The system restarts after the camera is turned on and off continuously for a period of time . Open camera command is “gst-launch-1.0 -v v4l2src device=/dev/video1 ! fakesink” .
The attachment is the test script and kernel log . The kernel error start on line 14152.

[ 3840.652835] TI960-1 0x1 = 0x0[ 3840.818873] CPU4: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000200, esr=be000000
[ 3840.818879] CPU2: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000100, esr=be000000
[ 3840.818889] CPU7: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000301, esr=be000000
[ 3840.818893] CPU6: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000300, esr=be000000
[ 3840.818920] CPU1: SError detected, daif=140, spsr=0x40400145, mpidr=80000001, esr=be000000
[ 3840.818949] **************************************
[ 3840.818951] * For more Internal Decode Help
[ 3840.818952] * http://nv/cbberr
[ 3840.818954] * NVIDIA userID is required to access
[ 3840.818955] **************************************
[ 3840.818957] CPU:2, Error:RCE-NOC
[ 3840.818961] Error Logger : 1
[ 3840.818971] ErrLog0 : 0x80030600
[ 3840.818973] Transaction Type : RD - Read, Incrementing
[ 3840.818978] CPU3: SError detected, daif=140, spsr=0x40400045, mpidr=80000101, esr=be000000
[ 3840.818979] Error Code : TMO
[ 3840.818981] Error Source : Target NIU
[ 3840.818983] Error Description : Target time-out error
[ 3840.818986] Packet header Lock : 0
[ 3840.818987] Packet header Len1 : 3
[ 3840.818989] NOC protocol version : version >= 2.7
[ 3840.818994] CPU:0, Error:CBB-NOC@0x2300000,irq=486
[ 3840.818996] ErrLog1 : 0x157600
[ 3840.818998] **************************************
[ 3840.818999] ErrLog2 : 0x0
[ 3840.819001] * For more Internal Decode Help
[ 3840.819003] RouteId : 0x157600
[ 3840.819006] TI960-1 0x20 = 0x0
[ 3840.819007] * http://nv/cbberr
[ 3840.819009] InitFlow : cpu_p_i/I/0
[ 3840.819011] * NVIDIA userID is required to access
[ 3840.819012] Targflow : cbb_t/T/0
[ 3840.819014] **************************************
[ 3840.819015] TargSubRange : 27
[ 3840.819017] CPU:0, Error:CBB-NOC
[ 3840.819018] SeqId : 0
[ 3840.819020] Error Logger : 1
[ 3840.819023] ErrLog3 : 0x5c00414
[ 3840.819024] ErrLog4 : 0x0
[ 3840.819028] ErrLog0 : 0x80030600
[ 3840.819030] Transaction Type : RD - Read, Incrementing
[ 3840.819031] Error Code : TMO
[ 3840.819033] Error Source : Target NIU
[ 3840.819034] Error Description : Target time-out error
[ 3840.819036] Packet header Lock : 0
[ 3840.819039] Packet header Len1 : 3
[ 3840.819040] NOC protocol version : version >= 2.7
[ 3840.819042] ErrLog1 : 0x9528aa
[ 3840.819043] ErrLog2 : 0x0
[ 3840.819044] RouteId : 0x9528aa
[ 3840.819046] InitFlow : rce_p2ps/I/rce_p2ps
[ 3840.819048] Targflow : host1x_p2pm/T/host1x_p2pm
[ 3840.819050] TargSubRange : 20
[ 3840.819051] SeqId : 0
[ 3840.819052] ErrLog3 : 0x414
[ 3840.819054] ErrLog4 : 0x0
[ 3840.819058] Address : 0x15c00414 (unknown device)
[ 3840.819060] ErrLog5 : 0x387e31
[ 3840.819065] Master ID : RCE
[ 3840.819067] Security Group(GRPSEC): 0x3f
[ 3840.819068] Cache : 0x1 – Device
[ 3840.819070] Protection : 0x3 – Privileged, Non-Secure, Data Access
[ 3840.819072] FALCONSEC : 0x0
[ 3840.819073] Virtual Queuing Channel(VQC): 0x0
[ 3840.819077] **************************************
[ 3840.819079] Address : 0x15c00414 (unknown device)
[ 3840.819081] ErrLog5 : 0x2af0fc71
[ 3840.819082] Non-Modify : 0x1
[ 3840.819084] AXI ID : 0x55
[ 3840.819085] Master ID : RCE
[ 3840.819087] Security Group(GRPSEC): 0x3f
[ 3840.819089] Cache : 0x1 – Device
[ 3840.819090] **************************************
[ 3840.819093] Protection : 0x3 – Privileged, Non-Secure, Data Access
[ 3840.819094] * For more Internal Decode Help
[ 3840.819095] FALCONSEC : 0x0
[ 3840.819097] * http://nv/cbberr
[ 3840.819098] Virtual Queuing Channel(VQC): 0x0
[ 3840.819100] * NVIDIA userID is required to access
[ 3840.819101] **************************************

AutoExTest_disp.sh (701 Bytes)
1.txt (733.0 KB)

3

hello 525554536,

it looks like a long run test.
may I know how many test cycles, or, how long can you reproduce such failure?

BTW,
we had seen this SError detected, failure, it’s timeout for accessing register.
since you’re based-on JP4.4, is it possible for moving to later JetPack release version for confirmation?

4

The time is not fixed, most of them are from one and a half hours to two and a half hours for testing; Sometimes within an hour. The camera is SG2-AR0231C-0202-FPDLink, auto trigger.

There is no way to upgrade to the latest jetpack. This is the customer’s test environment, and they cannot update the version at present.

BTW.
Is that version sure to fix this problem.

5

hello 525554536,

may I also confirm is this a developer kit, or a customize board?
I saw some fixes check-in to rel-32 code-line for SError detected failure, however, it seems related to display driver side.

6

We used customize board, and no screen used. This test only capture video, not view. You can try it on developer kit, this problem will be common.
If enable trace for more information , this error will not easy to reproduce.

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace

When we enable trace, the system has been running normally for more than 4 hours.

9

hello 525554536,

did you meant you also see the same failure on devkit?
we’ve try running your test script AutoExTest_disp.sh about 5 hours, cannot see the issue on the latest release version.

10

which release package used? r35.1?

11

hello 525554536,

yap, we’re confirmed with JP-5.0.2 (l4t-r35.1) on AGX Xavier.

12

Whether can perform the test on jetpack 4.4; if reproduce , provide the fault causes and how to fix it on latest release version. We want to make sure whether it is the kernel fault or our camera driver fault. We lack the conditions for the devkit to test the camera drivers.

15

hello 525554536,

FYI, we had further do the test with JetPack-4.6.3 release version.

we tried running your test script with USB cameras (since you’re running with v4l2src plugin). it’s confirmed we cannot repo the same failure on Jetson AGX Xavier with JetPack-4.6.3 release.

16

We have reproduce the probelm on jetPack 5.0.2 on our board, when we excute AutoExTest_diso.sh about 49000 seconds.

[19018.955853] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[19021.535372] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[19021.535632] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[19021.536832] (NULL device *): vi_capture_control_message: NULL VI channel received
[19021.537005] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[19021.537300] (NULL device *): vi_capture_control_message: NULL VI channel received
[19021.537462] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 1
[19021.537983] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[26247.010431] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[26247.055803] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[26249.695360] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[26249.695605] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[26249.696820] (NULL device *): vi_capture_control_message: NULL VI channel received
[26249.697045] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[26249.697272] (NULL device *): vi_capture_control_message: NULL VI channel received
[26249.697439] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 2
[26249.697999] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[26926.860536] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[26926.906778] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[26929.631358] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[26929.631648] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[26929.632835] (NULL device *): vi_capture_control_message: NULL VI channel received
[26929.633017] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[26929.633235] (NULL device *): vi_capture_control_message: NULL VI channel received
[26929.633397] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 1
[26929.633951] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[39406.236692] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[39406.236750] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[39406.236941] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[39406.237206] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[39406.706829] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[39406.955796] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[39409.375347] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[39409.375655] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[39409.376979] (NULL device *): vi_capture_control_message: NULL VI channel received
[39409.377180] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[39409.377376] (NULL device *): vi_capture_control_message: NULL VI channel received
[39409.377520] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 2
[39409.378037] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[39409.631359] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[39409.631623] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[39409.632869] (NULL device *): vi_capture_control_message: NULL VI channel received
[39409.633056] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[39409.633275] (NULL device *): vi_capture_control_message: NULL VI channel received
[39409.633475] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 1
[39409.633995] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[43360.860519] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[43360.906821] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[43363.551344] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[43363.551617] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[43363.552839] (NULL device *): vi_capture_control_message: NULL VI channel received
[43363.553056] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[43363.553277] (NULL device *): vi_capture_control_message: NULL VI channel received
[43363.553444] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 1
[43363.554022] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[46943.555815] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[46946.271368] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[46946.271648] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[46946.272844] (NULL device *): vi_capture_control_message: NULL VI channel received
[46946.272999] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[46946.273234] (NULL device *): vi_capture_control_message: NULL VI channel received
[46946.273372] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 1
[46946.274393] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[48858.560449] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[48858.605848] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[48861.151338] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[48861.151633] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[48861.152843] (NULL device *): vi_capture_control_message: NULL VI channel received
[48861.153010] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[48861.153238] (NULL device *): vi_capture_control_message: NULL VI channel received
[48861.153415] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 2
[48861.153945] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
��BUG: camera-ip/vi5/vi5.c:415 [vi5_check_falcon_failure] "VI FALCON FAILURE: 0x40000000"
[ 49003.850876] Camera-FW on t194-rce-safe started
TCU early console enabled.
[ 49003.918155] Camera-FW on t194-rce-safe ready SHA1=d48f1e27 (crt 0.773 ms, total boot 68.081 ms)
��[48984.031369] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[48984.031383] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[48984.031403] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[48984.031629] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[48984.032340] tegra194-vi5 15c10000.vi: vi_capture_release: control failed, errno 1
[48984.032365] tegra194-vi5 15c10000.vi: vi_capture_release: control failed, errno 1
[48984.032789] video4linux video2: vi capture release failed
[48984.032821] video4linux video1: vi capture release failed
[48984.032988] tegra-camrtc-capture-vi tegra-capture-vi: fatal: error recovery failed
[48984.033039] tegra-camrtc-capture-vi tegra-capture-vi: fatal: error recovery failed
[49012.212858] ------------[ cut here ]------------
[49012.213013] WARNING: CPU: 2 PID: 67490 at kernel/kthread.c:459 __kthread_bind_mask+0x34/0x90
[49012.213243] WARNING: CPU: 1 PID: 67493 at lib/refcount.c:25 refcount_warn_saturate+0x98/0x140
[49012.215837] ---[ end trace b6f35c23f6811aad ]---
[49012.216112] ---[ end trace b6f35c23f6811aae ]---
[49012.216318] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[49012.216640] Mem abort info:
[49012.216742]   ESR = 0x96000044
[49012.216833]   EC = 0x25: DABT (current EL), IL = 32 bits
[49012.216996]   SET = 0, FnV = 0
[49012.217095]   EA = 0, S1PTW = 0
[49012.217179] Data abort info:
[49012.217300]   ISV = 0, ISS = 0x00000044
[49012.217322] WARNING: CPU: 5 PID: 67493 at kernel/kthread.c:459 __kthread_bind_mask+0x34/0x90
[49012.217417]   CM = 0, WnR = 1
[49012.217741] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000119ab1000
[49012.217915] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000
[49012.218133]  lzo_rle lzo_compress
[49012.218157] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[49012.218162]  can_dev can_raw
[49012.218284] Modules linked in: input_leds
[49012.219881]  zram can
[49012.222509]  fuse mttcan
[49012.226964]  mv88e6xxx loop
[49012.228987]  lzo_rle lzo_compress
[49012.231519]  snd_soc_tegra210_ope snd_soc_tegra186_dspk
[49012.234320]  can_dev can_raw
[49012.238000]  snd_soc_tegra186_asrc
[49012.242978]  zram can mv88e6xxx
[49012.245867]  snd_soc_tegra186_arad snd_soc_tegra210_iqc
[49012.249020]  loop snd_soc_tegra210_ope
[49012.252088]  snd_soc_tegra210_mvc
[49012.257337]  snd_soc_tegra186_dspk snd_soc_tegra186_asrc
[49012.261098]  snd_soc_tegra210_admaif snd_soc_tegra210_afc
[49012.264348]  snd_soc_tegra186_arad snd_soc_tegra210_iqc
[49012.269758]  snd_soc_tegra210_adx snd_soc_tegra210_dmic snd_soc_tegra210_i2s
[49012.275534]  snd_soc_tegra210_mvc snd_soc_tegra210_admaif
[49012.280691]  snd_soc_tegra210_amx snd_soc_tegra210_sfc
[49012.287346]  snd_soc_tegra210_afc
[49012.292941]  snd_soc_tegra_pcm snd_soc_tegra210_mixer
[49012.298016]  snd_soc_tegra210_adx snd_soc_tegra210_dmic
[49012.301081]  nvgpu aes_ce_blk
[49012.306502]  snd_soc_tegra210_i2s snd_soc_tegra210_amx
[49012.311494]  crypto_simd cryptd
[49012.314643]  snd_soc_tegra210_sfc snd_soc_tegra_pcm
[49012.319896]  aes_ce_cipher ghash_ce
[49012.323302]  snd_soc_tegra210_mixer nvgpu
[49012.328147]  sha2_ce sha256_arm64
[49012.331446]  aes_ce_blk crypto_simd
[49012.335645]  sha1_ce snd_hda_codec_hdmi
[49012.338882]  cryptd aes_ce_cipher
[49012.342470]  snd_soc_tegra210_adsp snd_soc_tegra_machine_driver
[49012.346663]  ghash_ce sha2_ce
[49012.349849]  snd_soc_tegra_utils
[49012.355858]  sha256_arm64 sha1_ce
[49012.359175]  snd_hda_tegra snd_soc_simple_card_utils
[49012.362331]  snd_hda_codec_hdmi snd_soc_tegra210_adsp
[49012.365914]  snd_soc_spdif_tx pwm_fan
[49012.370560]  snd_soc_tegra_machine_driver snd_soc_tegra_utils
[49012.375889]  snd_hda_codec snd_soc_rt5659
[49012.379563]  snd_hda_tegra snd_soc_simple_card_utils
[49012.385163]  nvadsp snd_soc_tegra210_ahub
[49012.389276]  snd_soc_spdif_tx pwm_fan
[49012.394175]  nct1008 max77620_thermal
[49012.398204]  snd_hda_codec snd_soc_rt5659
[49012.401445]  snd_soc_rl6231 ucsi_ccg
[49012.405288]  nvadsp snd_soc_tegra210_ahub
[49012.409087]  tegra210_adma typec_ucsi
[49012.412643]  nct1008 max77620_thermal
[49012.416756]  typec ina3221
[49012.420715]  snd_soc_rl6231
[49012.424276]  tegra_bpmp_thermal userspace_alert
[49012.426731]  ucsi_ccg tegra210_adma
[49012.429443]  snd_hda_core spidev
[49012.434081]  typec_ucsi typec
[49012.437755]  spi_tegra114 ar0231(E)
[49012.441252]  ina3221 tegra_bpmp_thermal
[49012.444400]  binfmt_misc nvmap
[49012.447861]  userspace_alert snd_hda_core
[49012.451757]  ip_tables x_tables
[49012.454733]  spidev spi_tegra114
[49012.459192] 
[49012.462339]  ar0231(E) binfmt_misc
[49012.465603] CPU: 5 PID: 67493 Comm: gmain Tainted: G        W   E     5.10.104-tegra #40
[49012.465612] Hardware name: Unknown Jetson-AGX/Jetson-AGX, BIOS 1.0-d7fb19b 08/10/2022
[49012.467151]  nvmap ip_tables
[49012.470322] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--)
[49012.470354] pc : __kthread_bind_mask+0x34/0x90
[49012.478269]  x_tables
[49012.485884] lr : __kthread_bind_mask+0x30/0x90
[49012.485893] sp : ffff8000317d3a40
[49012.488791] CPU: 2 PID: 67490 Comm: v4l2src0:src Tainted: G        W   E     5.10.104-tegra #40
[49012.488800] Hardware name: Unknown Jetson-AGX/Jetson-AGX, BIOS 1.0-d7fb19b 08/10/2022
[49012.494891] x29: ffff8000317d3a40 x28: ffff6f80caee8000 
[49012.499281] pstate: 60400089 (nZCv daIf +PAN -UAO -TCO BTYPE=--)
[49012.499301] pc : __prepare_to_swait+0x50/0x60
[49012.501197] 
[49012.501208] x27: ffffcf52863d7000 
[49012.505745] lr : __prepare_to_swait+0x1c/0x60
[49012.505755] sp : ffff80003183ba10
[49012.508978] x26: 0000ffff9b0dff0c 
[49012.517465] x29: ffff80003183ba10 x28: ffff6f8005872b80 
[49012.525087] x25: fffffffffffffdfe 
[49012.530413] 
[49012.530431] x27: ffffcf52863d7000 
[49012.536367] x24: ffff6f80caee8600 
[49012.540827] x26: 0000ffffa91b2018 
[49012.542403] x23: ffff6f800361e1f8 x22: ffff6f875583e0a0 
[49012.545558] x25: fffffffffffffdfe x24: ffff6f8005872b80 
[49012.550277] 
[49012.550287] x21: ffffcf5285ba74b0 
[49012.553430] 
[49012.553448] x23: 0000000000000002 
[49012.556880] x20: 0000000000000040 
[49012.562642] x22: ffff6f8005872b80 
[49012.566027] x19: ffff6f804d444880 x18: 0000000000000010 
[49012.567512] 
[49012.567522] x21: 7fffffffffffffff 
[49012.570667] 
[49012.570684] x17: 0000000000000000 
[49012.574084] x20: ffff6f80bdfd9fd0 
[49012.577665] x16: 0000000000000000 
[49012.582919] x19: ffff80003183ba70 x18: 0000000000000000 
[49012.588517] x15: ffff6f80caee8570 x14: 656b207461203039 
[49012.590088] 
[49012.590100] x17: 0000000000000007 
[49012.593261] 
[49012.595076] x16: 000000000000000e 
[49012.598062] x13: 343736203a444950 x12: 343a632e64616572 
[49012.601477] x15: 0000000000000361 x14: 00000000000001d8 
[49012.605137] 
[49012.605151] x11: 68746b2f6c656e72 
[49012.610220] 
[49012.610234] x13: 0000000000000003 
[49012.611834] x10: 6e69625f64616572 
[49012.615211] x12: 0000000000000500 
[49012.616694] 
[49012.616708] x9 : 68746b5f5f203935 
[49012.620309] x11: 0000000000000001 x10: 0000000000000a80 
[49012.623433] x8 : 2032203a55504320 
[49012.627101] 
[49012.627111] x9 : ffff80003183b8c0 
[49012.632183] x7 : 3a474e494e524157 x6 : 000000000000000a 
[49012.637608] x8 : ffff6f8005873660 
[49012.639209] 
[49012.639220] x5 : ffff6f877fd78958 
[49012.642851] x7 : 0000000000000004 x6 : 0000000000000000 
[49012.644428] x4 : 0000000000000020 
[49012.647580] 
[49012.647591] x5 : 000000004e0f0040 
[49012.653178] x3 : 0000000000000000 x2 : 6d2ab1b536ebae00 
[49012.658431] x4 : 0000000000000020 
[49012.659744] 
[49012.659755] x1 : 0000000000000000 
[49012.663071] x3 : 0000000000000000 x2 : 0000000000000000 
[49012.664678] x0 : 0000000000000000 
[49012.668056] 
[49012.668066] x1 : ffff80003183ba78 
[49012.671726] 
[49012.671767] Call trace:
[49012.675140] x0 : ffff6f80bdfd9fd8 
[49012.676551]  __kthread_bind_mask+0x34/0x90
[49012.676563]  kthread_unpark+0xa0/0xb0
[49012.679607] Call trace:
[49012.679625]  __prepare_to_swait+0x50/0x60
[49012.685039]  kthread_stop+0x98/0x260
[49012.685056]  vi5_channel_stop_kthreads+0x40/0x60
[49012.688707]  wait_for_completion+0x74/0x110
[49012.688746]  kthread_stop+0xa8/0x260
[49012.690281]  vi5_channel_stop_streaming+0xb4/0xd0
[49012.690296]  tegra_channel_stop_streaming+0x38/0x60
[49012.693439]  vi5_channel_stop_kthreads+0x40/0x60
[49012.693451]  vi5_channel_stop_streaming+0xb4/0xd0
[49012.698817]  __vb2_queue_cancel+0x3c/0x220
[49012.698832]  vb2_core_queue_release+0x2c/0x60
[49012.702450]  tegra_channel_stop_streaming+0x38/0x60
[49012.702497]  __vb2_queue_cancel+0x3c/0x220
[49012.703591]  _vb2_fop_release+0x84/0xa0
[49012.703605]  tegra_channel_close+0x64/0x180
[49012.706915]  vb2_core_queue_release+0x2c/0x60
[49012.706929]  _vb2_fop_release+0x84/0xa0
[49012.712597]  v4l2_release+0xbc/0xf0
[49012.712614]  __fput+0x7c/0x230
[49012.715753]  tegra_channel_close+0x64/0x180
[49012.715799]  v4l2_release+0xbc/0xf0
[49012.717324]  ____fput+0x20/0x30
[49012.717340]  task_work_run+0x88/0x140
[49012.720824]  __fput+0x7c/0x230
[49012.720837]  ____fput+0x20/0x30
[49012.725814]  do_exit+0x350/0xab0
[49012.725828]  do_group_exit+0x48/0xb0
[49012.729488]  task_work_run+0x88/0x140
[49012.729503]  do_exit+0x350/0xab0
[49012.731001]  get_signal+0x104/0x830
[49012.731019]  do_notify_resume+0x17c/0x9f0
[49012.734128]  do_group_exit+0x48/0xb0
[49012.734170]  get_signal+0x104/0x830
[49012.739896]  work_pending+0xc/0x384
[49012.739912] ---[ end trace b6f35c23f6811aaf ]---
[49012.742966]  do_notify_resume+0x17c/0x9f0
[49012.742980]  work_pending+0xc/0x384
[49012.745730] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[49012.747956] Code: f9400a82 f9000a81 91002280 a9008a60 (f9000041) 
[49012.749524] Mem abort info:
[49012.749539]   ESR = 0x96000044
[49012.751988] ---[ end trace b6f35c23f6811ab0 ]---
[49012.755643]   EC = 0x25: DABT (current EL), IL = 32 bits
[49012.755653]   SET = 0, FnV = 0
[49012.759761] Kernel panic - not syncing: Oops: Fatal exception
[49012.763428]   EA = 0, S1PTW = 0
[49012.763439] Data abort info:
[49012.765887] SMP: stopping secondary CPUs
[49012.769990]   ISV = 0, ISS = 0x00000044
[49012.770003]   CM = 0, WnR = 1
[49012.966261] user pgtable: 4k pages, 48-bit VAs, pgdp=000000014a161000
[49012.972815] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000
[49014.013220] SMP: failed to stop secondary CPUs 2,5
[49014.013361] Kernel Offset: 0x4f52748c0000 from 0xffff800010000000
[49014.013501] PHYS_OFFSET: 0xffff908100000000
[49014.013598] CPU features: 0x8240002,03802a30
[49014.013702] Memory Limit: none
[49014.013782] ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

jetpcak5.0.2_reboot_log.txt (189.3 KB)

17

hello 525554536,

the error is different now for JP-5.0.2
there’s an error, NULL VI channel received. which means the camera device did not register to linux kernel correctly.
could you please confirm the video node, i.e. /dev/video* still exist when you trying to access the stream.

18

The videoN device is always exist. It is register on the camera driver loading. This video can view by cmd： gst-launch-1.0 -v v4l2src device=/dev/videoN ! fakesink

19

as I mentioned in comment #15.
we don’t have that FPDLink Ser/Des camera board. we’ve test v4l2src plugin with USB cameras, we cannot reproduce issues by running the script.

since we’re running the same plugin, it’s using the same camera software stack.

20

The command “sudo jetson_clocks” talk in this link appears to be valid. jetpack5.0.2 is runing more than24 hours; and jetpack 4.4 is running more than16 hours; and they are still running.

If jetson_clocks useful, Can analyze the fault cause and logic?

21

hello 525554536,

may I know how many cameras on your customize board.
this script controls CPU/GPU/Memory clocks, it check max available rate and overwrite its clock settings.

this is system level configurations,
I am wondering if you saw different failure rate with/without enabling VI tracing logs even not running jetson_clocks utility.
BTW,
you may see-also Power Mode Controls, it’s able to change power mode with nvpmodel command or, using the nvpmodel GUI.

22

2 camera used.
If enabling VI tracing logs event ,system will running longer.
The power mode is set to MAX.

23

hello 525554536,

okay, since that’s customize board, and we cannot reproduce the same failure locally.
please have system level configurations for your solution.

25

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.