"fence timeout on [ffffffc09f022300] after 1500ms" after 15 days running on IMX327

We have developed H/W and S/W that obtains V4L2_PIX_FMT_YUV420M data of IMX327 through libargus and compresses it with NVENC.

R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t210ref, EABI: aarch64, DATE: Mon Jul 26 19:20:30 UTC 2021

After normal operation for about 15 days, data transmission from libargus stopped with the message below.
(Attach the entire message as a file.)
jetson_imx327_libargus_blocked.log (355.6 KB)

Which part should be analyzed?

[2022-05-17 16:55:10.075]  5μ›” 17 16:55:08 imx327-desktop kernel: fence timeout on [ffffffc09f022300] after 1500ms
[2022-05-17 16:55:10.075]  5μ›” 17 16:55:08 imx327-desktop kernel: name=[nvhost_sync:34], current value=36899984 waiting value=36899985
[2022-05-17 16:55:10.090]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- mlocks ----
[2022-05-17 16:55:10.096]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.096]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- syncpts ----
[2022-05-17 16:55:10.096]  5μ›” 17 16:55:08 imx327-desktop kernel: id 1 (disp0_a) min 21700 max 21700 refs 1 (previous client : )
[2022-05-17 16:55:10.113]  5μ›” 17 16:55:08 imx327-desktop kernel: id 2 (disp0_b) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:10.130]  5μ›” 17 16:55:08 imx327-desktop kernel: fence timeout on [ffffffc0f10f8c00] after 1500ms
[2022-05-17 16:55:10.130]  5μ›” 17 16:55:08 imx327-desktop kernel: name=[nvhost_sync:13], current value=36948741 waiting value=36948742
[2022-05-17 16:55:10.130]  5μ›” 17 16:55:08 imx327-desktop kernel: id 3 (disp0_c) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:10.147]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- mlocks ----
[2022-05-17 16:55:10.147]  5μ›” 17 16:55:08 imx327-desktop kernel: id 8 (gm20b_507) min 46410 max 46410 refs 1 (previous client : )
[2022-05-17 16:55:10.163]  5μ›” 17 16:55:08 imx327-desktop kernel: fence timeout on [ffffffc0f10f80c0] after 1500ms
[2022-05-17 16:55:10.163]  5μ›” 17 16:55:08 imx327-desktop kernel: id 9 (gm20b_506) min 236 max 236 refs 1 (previous client : )
[2022-05-17 16:55:10.180]  5μ›” 17 16:55:08 imx327-desktop kernel: name=[nvhost_sync:14], current value=36916609 waiting value=36916610
[2022-05-17 16:55:10.180]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.195]  5μ›” 17 16:55:08 imx327-desktop kernel: id 11 (gm20b_505) min 287672 max 287672 refs 1 (previous client : gm20b_505)
[2022-05-17 16:55:10.195]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- syncpts ----
[2022-05-17 16:55:10.212]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- mlocks ----
[2022-05-17 16:55:10.213]  5μ›” 17 16:55:08 imx327-desktop kernel: id 12 (54340000.vic_0) min 16 max 16 refs 1 (previous client : gm20b_504)
[2022-05-17 16:55:10.213]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.230]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- syncpts ----
[2022-05-17 16:55:10.230]  5μ›” 17 16:55:08 imx327-desktop kernel: id 13 (54600000.isp_0) min 36948741 max 36948744 refs 4 (previous client : 54600000.isp_0)
[2022-05-17 16:55:10.246]  5μ›” 17 16:55:08 imx327-desktop kernel: id 14 (54600000.isp_1) min 36916609 max 36916612 refs 4 (previous client : 54600000.isp_1)
[2022-05-17 16:55:10.246]  5μ›” 17 16:55:08 imx327-desktop kernel: id 15 (54600000.isp_2) min 113411995 max 113412002 refs 9 (previous client : 54600000.isp_2)
[2022-05-17 16:55:10.262]  5μ›” 17 16:55:08 imx327-desktop kernel: id 16 (54600000.isp_3) min 36899988 max 36899990 refs 4 (previous client : 54600000.isp_3)
[2022-05-17 16:55:10.279]  5μ›” 17 16:55:08 imx327-desktop kernel: id 17 (54680000.isp_0) min 3018 max 3018 refs 1 (previous client : 54680000.isp_0)
[2022-05-17 16:55:10.279]  5μ›” 17 16:55:08 imx327-desktop kernel: id 18 (gm20b_499) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:10.296]  5μ›” 17 16:55:08 imx327-desktop kernel: id 22 (gm20b_504) min 704 max 704 refs 1 (previous client : )
[2022-05-17 16:55:10.296]  5μ›” 17 16:55:08 imx327-desktop kernel: id 23 (gm20b_503) min 54 max 54 refs 1 (previous client : )
[2022-05-17 16:55:10.313]  5μ›” 17 16:55:08 imx327-desktop kernel: id 24 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:10.329]  5μ›” 17 16:55:08 imx327-desktop kernel: id 25 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:10.330]  5μ›” 17 16:55:08 imx327-desktop kernel: id 26 (vblank0) min 1045339 max -42 refs 1 (previous client : )
[2022-05-17 16:55:10.346]  5μ›” 17 16:55:08 imx327-desktop kernel: id 28 (gm20b_500) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:10.346]  5μ›” 17 16:55:08 imx327-desktop kernel: id 29 (54080000.vi_0) min 36899983 max 36899986 refs 4 (previous client : 54080000.vi_0)
[2022-05-17 16:55:10.363]  5μ›” 17 16:55:08 imx327-desktop kernel: id 31 (54080000.vi_2) min 73858634 max 73858639 refs 7 (previous client : 54080000.vi_2)
[2022-05-17 16:55:10.363]  5μ›” 17 16:55:08 imx327-desktop kernel: id 34 (54080000.vi_4) min 36899984 max 36899986 refs 4 (previous client : 54080000.vi_4)
[2022-05-17 16:55:10.380]  5μ›” 17 16:55:08 imx327-desktop kernel: id 35 (54340000.vic_0) min 333303033 max 333303033 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.396]  5μ›” 17 16:55:08 imx327-desktop kernel: id 37 (gm20b_495) min 18 max 18 refs 1 (previous client : gm20b_498)
[2022-05-17 16:55:10.396]  5μ›” 17 16:55:08 imx327-desktop kernel: id 38 (gm20b_493) min 20 max 20 refs 1 (previous client : gm20b_497)
[2022-05-17 16:55:10.413]  5μ›” 17 16:55:08 imx327-desktop kernel: id 39 (gm20b_494) min 18 max 18 refs 1 (previous client : gm20b_496)
[2022-05-17 16:55:10.430]  5μ›” 17 16:55:08 imx327-desktop kernel: id 40 (gm20b_498) min 18 max 18 refs 1 (previous client : gm20b_495)
[2022-05-17 16:55:10.430]  5μ›” 17 16:55:08 imx327-desktop kernel: id 41 (gm20b_496) min 18 max 18 refs 1 (previous client : gm20b_494)
[2022-05-17 16:55:10.446]  5μ›” 17 16:55:08 imx327-desktop kernel: id 42 (gm20b_497) min 18 max 18 refs 1 (previous client : gm20b_493)
[2022-05-17 16:55:10.446]  5μ›” 17 16:55:08 imx327-desktop kernel: id 44 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:10.463]  5μ›” 17 16:55:08 imx327-desktop kernel: id 46 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:10.480]  5μ›” 17 16:55:08 imx327-desktop kernel: id 48 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:10.480]  5μ›” 17 16:55:08 imx327-desktop kernel: id 51 (54380000.nvjpg_0) min 36899832 max 36899832 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.496]  5μ›” 17 16:55:08 imx327-desktop kernel: id 52 (54340000.vic_0) min 110699931 max 110699931 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.513]  5μ›” 17 16:55:08 imx327-desktop kernel: id 53 (54340000.vic_0) min 453 max 453 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.528]  5μ›” 17 16:55:08 imx327-desktop kernel: id 54 (54340000.vic_0) min 110699622 max 110699622 refs 1 (previous client : 54380000.nvjpg_0)
[2022-05-17 16:55:10.529]  5μ›” 17 16:55:08 imx327-desktop kernel: id 55 (54340000.vic_0) min 110699910 max 110699910 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.546]  5μ›” 17 16:55:08 imx327-desktop kernel: id 56 (54340000.vic_0) min 110699928 max 110699928 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.563]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.563]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- channels ----
[2022-05-17 16:55:10.563]  5μ›” 17 16:55:08 imx327-desktop kernel: id 1 (disp0_a) min 21700 max 21700 refs 1 (previous client : )
[2022-05-17 16:55:10.580]  5μ›” 17 16:55:08 imx327-desktop kernel: id 2 (disp0_b) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:10.580]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.580]                                          channel 0 - 54600000.isp
[2022-05-17 16:55:10.596]  5μ›” 17 16:55:08 imx327-desktop kernel: id 3 (disp0_c) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:10.596]  5μ›” 17 16:55:08 imx327-desktop kernel: 0-54600000.isp (16): 
[2022-05-17 16:55:10.613]  5μ›” 17 16:55:08 imx327-desktop kernel: id 8 (gm20b_507) min 46410 max 46410 refs 1 (previous client : )
[2022-05-17 16:55:10.613]  5μ›” 17 16:55:08 imx327-desktop kernel: id 9 (gm20b_506) min 236 max 236 refs 1 (previous client : )
[2022-05-17 16:55:10.630]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000000d
[2022-05-17 16:55:10.630]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000008, DMAGET 00000fb0, DMACTL 00000000
[2022-05-17 16:55:10.646]  5μ›” 17 16:55:08 imx327-desktop kernel: id 11 (gm20b_505) min 287672 max 287672 refs 1 (previous client : gm20b_505)
[2022-05-17 16:55:10.646]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000000d, CBSTAT 00010050
[2022-05-17 16:55:10.663]  5μ›” 17 16:55:08 imx327-desktop kernel: id 12 (54340000.vic_0) min 16 max 16 refs 1 (previous client : gm20b_504)
[2022-05-17 16:55:10.663]  5μ›” 17 16:55:08 imx327-desktop kernel: id 13 (54600000.isp_0) min 36948741 max 36948744 refs 4 (previous client : 54600000.isp_0)
[2022-05-17 16:55:10.680]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.680]                                          ffffffc0bb6cb000: JOB, syncpt_id=13, syncpt_val=36948742, first_get=00000f18, timeout=10000, num_slots=3, num_handles=1
[2022-05-17 16:55:10.695]  5μ›” 17 16:55:08 imx327-desktop kernel: id 14 (54600000.isp_1) min 36916609 max 36916612 refs 4 (previous client : 54600000.isp_1)
[2022-05-17 16:55:10.713]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2b8c, 8 words
[2022-05-17 16:55:10.713]  5μ›” 17 16:55:08 imx327-desktop kernel: id 15 (54600000.isp_2) min 113411995 max 113412002 refs 9 (previous client : 54600000.isp_2)
[2022-05-17 16:55:10.730]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:10.730]  5μ›” 17 16:55:08 imx327-desktop kernel: id 16 (54600000.isp_3) min 36899988 max 36899990 refs 4 (previous client : 54600000.isp_3)
[2022-05-17 16:55:10.746]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000040d 
[2022-05-17 16:55:10.746]  5μ›” 17 16:55:08 imx327-desktop kernel: id 17 (54680000.isp_0) min 3018 max 3018 refs 1 (previous client : 54680000.isp_0)
[2022-05-17 16:55:10.763]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:10.763]  5μ›” 17 16:55:08 imx327-desktop kernel: id 18 (gm20b_499) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:10.778]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000050e 
[2022-05-17 16:55:10.779]  5μ›” 17 16:55:08 imx327-desktop kernel: id 22 (gm20b_504) min 704 max 704 refs 1 (previous client : )
[2022-05-17 16:55:10.779]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:10.796]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000610 
[2022-05-17 16:55:10.796]  5μ›” 17 16:55:08 imx327-desktop kernel: id 23 (gm20b_503) min 54 max 54 refs 1 (previous client : )
[2022-05-17 16:55:10.796]  5μ›” 17 16:55:08 imx327-desktop kernel: 200c0001 
[2022-05-17 16:55:10.813]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000005 
[2022-05-17 16:55:10.813]  5μ›” 17 16:55:08 imx327-desktop kernel: id 24 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:10.813]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.830]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2bac, 2 words
[2022-05-17 16:55:10.830]  5μ›” 17 16:55:08 imx327-desktop kernel: id 25 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:10.846]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:10.846]  5μ›” 17 16:55:08 imx327-desktop kernel: id 26 (vblank0) min 1045339 max -42 refs 1 (previous client : )
[2022-05-17 16:55:10.846]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:10.863]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.863]  5μ›” 17 16:55:08 imx327-desktop kernel: id 28 (gm20b_500) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:10.863]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.880]  5μ›” 17 16:55:08 imx327-desktop kernel: id 29 (54080000.vi_0) min 36899983 max 36899986 refs 4 (previous client : 54080000.vi_0)
[2022-05-17 16:55:10.880]  5μ›” 17 16:55:08 imx327-desktop kernel: id 31 (54080000.vi_2) min 73858634 max 73858639 refs 7 (previous client : 54080000.vi_2)
[2022-05-17 16:55:10.896]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.896]                                          channel 1 - 54080000.vi
[2022-05-17 16:55:10.912]  5μ›” 17 16:55:08 imx327-desktop kernel: 1-54080000.vi (12): 
[2022-05-17 16:55:10.912]  5μ›” 17 16:55:08 imx327-desktop kernel: id 34 (54080000.vi_4) min 36899984 max 36899986 refs 4 (previous client : 54080000.vi_4)
[2022-05-17 16:55:10.929]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000001d
[2022-05-17 16:55:10.929]  5μ›” 17 16:55:08 imx327-desktop kernel: id 35 (54340000.vic_0) min 333303033 max 333303033 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:10.946]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000050, DMAGET 00000050, DMACTL 00000000
[2022-05-17 16:55:10.946]  5μ›” 17 16:55:08 imx327-desktop kernel: id 37 (gm20b_495) min 18 max 18 refs 1 (previous client : gm20b_498)
[2022-05-17 16:55:10.963]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000001d, CBSTAT 00010050
[2022-05-17 16:55:10.963]  5μ›” 17 16:55:08 imx327-desktop kernel: id 38 (gm20b_493) min 20 max 20 refs 1 (previous client : gm20b_497)
[2022-05-17 16:55:10.978]  5μ›” 17 16:55:08 imx327-desktop kernel: id 39 (gm20b_494) min 18 max 18 refs 1 (previous client : gm20b_496)
[2022-05-17 16:55:10.978]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:10.994]                                          ffffffc0bb6c8800: JOB, syncpt_id=29, syncpt_val=36899984, first_get=00000f78, timeout=10000, num_slots=5, num_handles=1
[2022-05-17 16:55:10.999]  5μ›” 17 16:55:08 imx327-desktop kernel: id 40 (gm20b_498) min 18 max 18 refs 1 (previous client : gm20b_495)
[2022-05-17 16:55:11.015]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3674, 3 words
[2022-05-17 16:55:11.015]  5μ›” 17 16:55:08 imx327-desktop kernel: id 41 (gm20b_496) min 18 max 18 refs 1 (previous client : gm20b_494)
[2022-05-17 16:55:11.030]  5μ›” 17 16:55:08 imx327-desktop kernel: 304e0005 
[2022-05-17 16:55:11.030]  5μ›” 17 16:55:08 imx327-desktop kernel: id 42 (gm20b_497) min 18 max 18 refs 1 (previous client : gm20b_493)
[2022-05-17 16:55:11.046]  5μ›” 17 16:55:08 imx327-desktop kernel: 06c2879b 
[2022-05-17 16:55:11.046]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:11.046]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.063]  5μ›” 17 16:55:08 imx327-desktop kernel: id 44 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:11.063]  5μ›” 17 16:55:08 imx327-desktop kernel: id 46 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:11.080]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3680, 14 words
[2022-05-17 16:55:11.080]  5μ›” 17 16:55:08 imx327-desktop kernel: 124b0003 
[2022-05-17 16:55:11.096]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:11.096]  5μ›” 17 16:55:08 imx327-desktop kernel: id 48 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:11.113]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:11.113]  5μ›” 17 16:55:08 imx327-desktop kernel: id 51 (54380000.nvjpg_0) min 36899832 max 36899832 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.130]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:11.130]  5μ›” 17 16:55:08 imx327-desktop kernel: 12580003 
[2022-05-17 16:55:11.130]  5μ›” 17 16:55:08 imx327-desktop kernel: id 52 (54340000.vic_0) min 110699931 max 110699931 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.146]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:11.146]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:11.146]  5μ›” 17 16:55:08 imx327-desktop kernel: id 53 (54340000.vic_0) min 453 max 453 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.163]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:11.163]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.178]  5μ›” 17 16:55:08 imx327-desktop kernel: id 54 (54340000.vic_0) min 110699622 max 110699622 refs 1 (previous client : 54380000.nvjpg_0)
[2022-05-17 16:55:11.178]  5μ›” 17 16:55:08 imx327-desktop kernel: 00001c1d 
[2022-05-17 16:55:11.196]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.196]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000522 
[2022-05-17 16:55:11.196]  5μ›” 17 16:55:08 imx327-desktop kernel: id 55 (54340000.vic_0) min 110699910 max 110699910 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.213]  5μ›” 17 16:55:08 imx327-desktop kernel: 20410001 
[2022-05-17 16:55:11.213]  5μ›” 17 16:55:08 imx327-desktop kernel: id 56 (54340000.vic_0) min 110699928 max 110699928 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.229]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000001 
[2022-05-17 16:55:11.229]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.229]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+36b8, 2 words
[2022-05-17 16:55:11.246]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.246]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000001f 
[2022-05-17 16:55:11.246]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.246]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.263]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.263]                                          ---- host general irq ----
[2022-05-17 16:55:11.263]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask_ext = 0xc0000000
[2022-05-17 16:55:11.280]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask = 0x80000000
[2022-05-17 16:55:11.280]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intc0mask = 0x00000001
[2022-05-17 16:55:11.280]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intmask = 0x00000011
[2022-05-17 16:55:11.296]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.296]                                          ---- host syncpt irq mask ----
[2022-05-17 16:55:11.296]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(0) = 0x54000001
[2022-05-17 16:55:11.313]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(1) = 0x44000001
[2022-05-17 16:55:11.313]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.313]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(2) = 0x00000010
[2022-05-17 16:55:11.329]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(3) = 0x00000000
[2022-05-17 16:55:11.329]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- channels ----
[2022-05-17 16:55:11.346]  5μ›” 17 16:55:08 imx327-desktop kernel: id 1 (disp0_a) min 21700 max 21700 refs 1 (previous client : )
[2022-05-17 16:55:11.346]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(4) = 0x00000000
[2022-05-17 16:55:11.362]  5μ›” 17 16:55:08 imx327-desktop kernel: id 2 (disp0_b) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:11.363]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.363]                                          channel 0 - 54600000.isp
[2022-05-17 16:55:11.379]  5μ›” 17 16:55:08 imx327-desktop kernel: id 3 (disp0_c) min 21 max 21 refs 1 (previous client : )
[2022-05-17 16:55:11.379]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(5) = 0x00000000
[2022-05-17 16:55:11.395]  5μ›” 17 16:55:08 imx327-desktop kernel: 0-54600000.isp (16): 
[2022-05-17 16:55:11.395]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(6) = 0x00000000
[2022-05-17 16:55:11.395]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000000d
[2022-05-17 16:55:11.413]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000008, DMAGET 00000fb0, DMACTL 00000000
[2022-05-17 16:55:11.413]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(7) = 0x00000000
[2022-05-17 16:55:11.429]  5μ›” 17 16:55:08 imx327-desktop kernel: id 8 (gm20b_507) min 46410 max 46410 refs 1 (previous client : )
[2022-05-17 16:55:11.429]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(8) = 0x00000000
[2022-05-17 16:55:11.446]  5μ›” 17 16:55:08 imx327-desktop kernel: id 9 (gm20b_506) min 236 max 236 refs 1 (previous client : )
[2022-05-17 16:55:11.446]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000000d, CBSTAT 00010050
[2022-05-17 16:55:11.462]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(9) = 0x00000000
[2022-05-17 16:55:11.462]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(10) = 0x00000000
[2022-05-17 16:55:11.462]  5μ›” 17 16:55:08 imx327-desktop kernel: id 11 (gm20b_505) min 287672 max 287672 refs 1 (previous client : gm20b_505)
[2022-05-17 16:55:11.479]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.479]                                          ffffffc0bb6cb000: JOB, syncpt_id=13, syncpt_val=36948742, first_get=00000f18, timeout=10000, num_slots=3, num_handles=1
[2022-05-17 16:55:11.496]  5μ›” 17 16:55:08 imx327-desktop kernel: id 12 (54340000.vic_0) min 16 max 16 refs 1 (previous client : gm20b_504)
[2022-05-17 16:55:11.513]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(11) = 0x00000000
[2022-05-17 16:55:11.513]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2b8c, 8 words
[2022-05-17 16:55:11.528]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.529]                                          ---- host syncpt irq status ----
[2022-05-17 16:55:11.529]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(0) = 0x00000000
[2022-05-17 16:55:11.545]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.545]  5μ›” 17 16:55:08 imx327-desktop kernel: id 13 (54600000.isp_0) min 36948741 max 36948744 refs 4 (previous client : 54600000.isp_0)
[2022-05-17 16:55:11.562]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000040d 
[2022-05-17 16:55:11.563]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.563]  5μ›” 17 16:55:08 imx327-desktop kernel: id 14 (54600000.isp_1) min 36916609 max 36916612 refs 4 (previous client : 54600000.isp_1)
[2022-05-17 16:55:11.580]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(1) = 0x00000000
[2022-05-17 16:55:11.580]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(2) = 0x00000000
[2022-05-17 16:55:11.596]  5μ›” 17 16:55:08 imx327-desktop kernel: id 15 (54600000.isp_2) min 113411995 max 113412002 refs 9 (previous client : 54600000.isp_2)
[2022-05-17 16:55:11.596]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000050e 
[2022-05-17 16:55:11.613]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.613]  5μ›” 17 16:55:08 imx327-desktop kernel: id 16 (54600000.isp_3) min 36899988 max 36899990 refs 4 (previous client : 54600000.isp_3)
[2022-05-17 16:55:11.630]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(3) = 0x00000000
[2022-05-17 16:55:11.630]  5μ›” 17 16:55:08 imx327-desktop kernel: id 17 (54680000.isp_0) min 3018 max 3018 refs 1 (previous client : 54680000.isp_0)
[2022-05-17 16:55:11.646]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000610 
[2022-05-17 16:55:11.646]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(4) = 0x00000000
[2022-05-17 16:55:11.646]  5μ›” 17 16:55:08 imx327-desktop kernel: 200c0001 
[2022-05-17 16:55:11.663]  5μ›” 17 16:55:08 imx327-desktop kernel: id 18 (gm20b_499) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:11.663]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000005 
[2022-05-17 16:55:11.680]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(5) = 0x00000000
[2022-05-17 16:55:11.680]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.680]  5μ›” 17 16:55:08 imx327-desktop kernel: id 22 (gm20b_504) min 704 max 704 refs 1 (previous client : )
[2022-05-17 16:55:11.695]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2bac, 2 words
[2022-05-17 16:55:11.695]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.695]                                          ---- host syncpt thresh ----
[2022-05-17 16:55:11.713]  5μ›” 17 16:55:08 imx327-desktop kernel: id 23 (gm20b_503) min 54 max 54 refs 1 (previous client : )
[2022-05-17 16:55:11.713]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(0) = 1
[2022-05-17 16:55:11.729]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:11.729]  5μ›” 17 16:55:08 imx327-desktop kernel: id 24 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:11.745]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:11.746]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.746]  5μ›” 17 16:55:08 imx327-desktop kernel: id 25 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[2022-05-17 16:55:11.762]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.762]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(13) = 36948742
[2022-05-17 16:55:11.762]  5μ›” 17 16:55:08 imx327-desktop kernel: id 26 (vblank0) min 1045339 max -42 refs 1 (previous client : )
[2022-05-17 16:55:11.780]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(14) = 36916610
[2022-05-17 16:55:11.780]  5μ›” 17 16:55:08 imx327-desktop kernel: id 28 (gm20b_500) min 6 max 6 refs 1 (previous client : )
[2022-05-17 16:55:11.796]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.796]                                          channel 1 - 54080000.vi
[2022-05-17 16:55:11.796]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(15) = 113411996
[2022-05-17 16:55:11.813]  5μ›” 17 16:55:08 imx327-desktop kernel: 1-54080000.vi (12): 
[2022-05-17 16:55:11.813]  5μ›” 17 16:55:08 imx327-desktop kernel: id 29 (54080000.vi_0) min 36899983 max 36899986 refs 4 (previous client : 54080000.vi_0)
[2022-05-17 16:55:11.830]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(16) = 36899989
[2022-05-17 16:55:11.830]  5μ›” 17 16:55:08 imx327-desktop kernel: id 31 (54080000.vi_2) min 73858634 max 73858639 refs 7 (previous client : 54080000.vi_2)
[2022-05-17 16:55:11.846]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000001d
[2022-05-17 16:55:11.846]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(29) = 36899984
[2022-05-17 16:55:11.863]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000050, DMAGET 00000050, DMACTL 00000000
[2022-05-17 16:55:11.863]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(31) = 73858635
[2022-05-17 16:55:11.879]  5μ›” 17 16:55:08 imx327-desktop kernel: id 34 (54080000.vi_4) min 36899984 max 36899986 refs 4 (previous client : 54080000.vi_4)
[2022-05-17 16:55:11.879]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(34) = 36899985
[2022-05-17 16:55:11.896]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000001d, CBSTAT 00010050
[2022-05-17 16:55:11.896]  5μ›” 17 16:55:08 imx327-desktop kernel: id 35 (54340000.vic_0) min 333303033 max 333303033 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:11.912]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.912]                                          ffffffc0bb6c8800: JOB, syncpt_id=29, syncpt_val=36899984, first_get=00000f78, timeout=10000, num_slots=5, num_handles=1
[2022-05-17 16:55:11.929]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3674, 3 words
[2022-05-17 16:55:11.929]  5μ›” 17 16:55:08 imx327-desktop kernel: id 37 (gm20b_495) min 18 max 18 refs 1 (previous client : gm20b_498)
[2022-05-17 16:55:11.946]  5μ›” 17 16:55:08 imx327-desktop kernel: 304e0005 
[2022-05-17 16:55:11.946]  5μ›” 17 16:55:08 imx327-desktop kernel: id 38 (gm20b_493) min 20 max 20 refs 1 (previous client : gm20b_497)
[2022-05-17 16:55:11.963]  5μ›” 17 16:55:08 imx327-desktop kernel: 06c2879b 
[2022-05-17 16:55:11.963]  5μ›” 17 16:55:08 imx327-desktop kernel: id 39 (gm20b_494) min 18 max 18 refs 1 (previous client : gm20b_496)
[2022-05-17 16:55:11.978]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:11.978]  5μ›” 17 16:55:08 imx327-desktop kernel: id 40 (gm20b_498) min 18 max 18 refs 1 (previous client : gm20b_495)
[2022-05-17 16:55:11.978]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:11.994]  5μ›” 17 16:55:08 imx327-desktop kernel: id 41 (gm20b_496) min 18 max 18 refs 1 (previous client : gm20b_494)
[2022-05-17 16:55:11.998]  5μ›” 17 16:55:08 imx327-desktop kernel: id 42 (gm20b_497) min 18 max 18 refs 1 (previous client : gm20b_493)
[2022-05-17 16:55:12.015]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3680, 14 words
[2022-05-17 16:55:12.015]  5μ›” 17 16:55:08 imx327-desktop kernel: 124b0003 
[2022-05-17 16:55:12.015]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:12.031]  5μ›” 17 16:55:08 imx327-desktop kernel: id 44 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:12.031]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:12.046]  5μ›” 17 16:55:08 imx327-desktop kernel: id 46 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:12.046]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:12.063]  5μ›” 17 16:55:08 imx327-desktop kernel: 12580003 
[2022-05-17 16:55:12.063]  5μ›” 17 16:55:08 imx327-desktop kernel: id 48 (544c0000.nvenc_0) min 36899975 max 36899975 refs 1 (previous client : 544c0000.nvenc_0)
[2022-05-17 16:55:12.080]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:12.080]  5μ›” 17 16:55:08 imx327-desktop kernel: id 51 (54380000.nvjpg_0) min 36899832 max 36899832 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:12.096]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:12.096]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:12.096]  5μ›” 17 16:55:08 imx327-desktop kernel: id 52 (54340000.vic_0) min 110699931 max 110699931 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:12.112]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.112]  5μ›” 17 16:55:08 imx327-desktop kernel: 00001c1d 
[2022-05-17 16:55:12.112]  5μ›” 17 16:55:08 imx327-desktop kernel: id 53 (54340000.vic_0) min 453 max 453 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:12.129]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.129]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000522 
[2022-05-17 16:55:12.146]  5μ›” 17 16:55:08 imx327-desktop kernel: 20410001 
[2022-05-17 16:55:12.146]  5μ›” 17 16:55:08 imx327-desktop kernel: id 54 (54340000.vic_0) min 110699622 max 110699622 refs 1 (previous client : 54380000.nvjpg_0)
[2022-05-17 16:55:12.163]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000001 
[2022-05-17 16:55:12.163]  5μ›” 17 16:55:08 imx327-desktop kernel: id 55 (54340000.vic_0) min 110699910 max 110699910 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:12.178]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.178]  5μ›” 17 16:55:08 imx327-desktop kernel: id 56 (54340000.vic_0) min 110699928 max 110699928 refs 1 (previous client : 54340000.vic_0)
[2022-05-17 16:55:12.195]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+36b8, 2 words
[2022-05-17 16:55:12.196]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.196]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000001f 
[2022-05-17 16:55:12.212]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.213]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.213]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.213]                                          ---- host general irq ----
[2022-05-17 16:55:12.213]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask_ext = 0xc0000000
[2022-05-17 16:55:12.230]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask = 0x80000000
[2022-05-17 16:55:12.230]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intc0mask = 0x00000001
[2022-05-17 16:55:12.246]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intmask = 0x00000011
[2022-05-17 16:55:12.246]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.246]                                          ---- host syncpt irq mask ----
[2022-05-17 16:55:12.263]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(0) = 0x54000001
[2022-05-17 16:55:12.263]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(1) = 0x44000001
[2022-05-17 16:55:12.263]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(2) = 0x00000010
[2022-05-17 16:55:12.279]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(3) = 0x00000000
[2022-05-17 16:55:12.279]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.279]  5μ›” 17 16:55:08 imx327-desktop kernel: ---- channels ----
[2022-05-17 16:55:12.296]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(4) = 0x00000000
[2022-05-17 16:55:12.296]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(5) = 0x00000000
[2022-05-17 16:55:12.313]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.313]                                          channel 0 - 54600000.isp
[2022-05-17 16:55:12.313]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(6) = 0x00000000
[2022-05-17 16:55:12.328]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(7) = 0x00000000
[2022-05-17 16:55:12.328]  5μ›” 17 16:55:08 imx327-desktop kernel: 0-54600000.isp (16): 
[2022-05-17 16:55:12.328]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(8) = 0x00000000
[2022-05-17 16:55:12.346]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(9) = 0x00000000
[2022-05-17 16:55:12.346]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000000d
[2022-05-17 16:55:12.363]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(10) = 0x00000000
[2022-05-17 16:55:12.363]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000008, DMAGET 00000fb0, DMACTL 00000000
[2022-05-17 16:55:12.379]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000000d, CBSTAT 00010050
[2022-05-17 16:55:12.379]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(11) = 0x00000000
[2022-05-17 16:55:12.379]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.396]                                          ---- host syncpt irq status ----
[2022-05-17 16:55:12.396]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(0) = 0x00000000
[2022-05-17 16:55:12.396]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.413]                                          ffffffc0bb6cb000: JOB, syncpt_id=13, syncpt_val=36948742, first_get=00000f18, timeout=10000, num_slots=3, num_handles=1
[2022-05-17 16:55:12.429]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(1) = 0x00000000
[2022-05-17 16:55:12.430]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2b8c, 8 words
[2022-05-17 16:55:12.430]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.446]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(2) = 0x00000000
[2022-05-17 16:55:12.446]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(3) = 0x00000000
[2022-05-17 16:55:12.463]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000040d 
[2022-05-17 16:55:12.463]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.463]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(4) = 0x00000000
[2022-05-17 16:55:12.478]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(5) = 0x00000000
[2022-05-17 16:55:12.478]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000050e 
[2022-05-17 16:55:12.478]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.495]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.496]                                          ---- host syncpt thresh ----
[2022-05-17 16:55:12.496]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000610 
[2022-05-17 16:55:12.496]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(0) = 1
[2022-05-17 16:55:12.513]  5μ›” 17 16:55:08 imx327-desktop kernel: 200c0001 
[2022-05-17 16:55:12.513]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000005 
[2022-05-17 16:55:12.513]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(13) = 36948742
[2022-05-17 16:55:12.529]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.529]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(14) = 36916610
[2022-05-17 16:55:12.529]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(15) = 113411996
[2022-05-17 16:55:12.546]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffdf0000+2bac, 2 words
[2022-05-17 16:55:12.546]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(16) = 36899989
[2022-05-17 16:55:12.563]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.563]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:12.563]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(29) = 36899984
[2022-05-17 16:55:12.579]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(31) = 73858635
[2022-05-17 16:55:12.579]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(34) = 36899985
[2022-05-17 16:55:12.596]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.596]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.596]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.596]                                          channel 1 - 54080000.vi
[2022-05-17 16:55:12.613]  5μ›” 17 16:55:08 imx327-desktop kernel: 1-54080000.vi (12): 
[2022-05-17 16:55:12.613]  5μ›” 17 16:55:08 imx327-desktop kernel: active class 01, offset 0050, val 0000001d
[2022-05-17 16:55:12.613]  5μ›” 17 16:55:08 imx327-desktop kernel: DMAPUT 00000050, DMAGET 00000050, DMACTL 00000000
[2022-05-17 16:55:12.628]  5μ›” 17 16:55:08 imx327-desktop kernel: CBREAD 0000001d, CBSTAT 00010050
[2022-05-17 16:55:12.628]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.644]                                          ffffffc0bb6c8800: JOB, syncpt_id=29, syncpt_val=36899984, first_get=00000f78, timeout=10000, num_slots=5, num_handles=1
[2022-05-17 16:55:12.646]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3674, 3 words
[2022-05-17 16:55:12.663]  5μ›” 17 16:55:08 imx327-desktop kernel: 304e0005 
[2022-05-17 16:55:12.663]  5μ›” 17 16:55:08 imx327-desktop kernel: 06c2879b 
[2022-05-17 16:55:12.663]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000000f 
[2022-05-17 16:55:12.678]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.679]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+3680, 14 words
[2022-05-17 16:55:12.679]  5μ›” 17 16:55:08 imx327-desktop kernel: 124b0003 
[2022-05-17 16:55:12.679]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:12.695]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:12.695]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:12.695]  5μ›” 17 16:55:08 imx327-desktop kernel: 12580003 
[2022-05-17 16:55:12.712]  5μ›” 17 16:55:08 imx327-desktop kernel: 00010000 
[2022-05-17 16:55:12.713]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000000 
[2022-05-17 16:55:12.713]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000004c 
[2022-05-17 16:55:12.713]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.729]  5μ›” 17 16:55:08 imx327-desktop kernel: 00001c1d 
[2022-05-17 16:55:12.729]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.729]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000522 
[2022-05-17 16:55:12.729]  5μ›” 17 16:55:08 imx327-desktop kernel: 20410001 
[2022-05-17 16:55:12.746]  5μ›” 17 16:55:08 imx327-desktop kernel: 00000001 
[2022-05-17 16:55:12.746]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.746]  5μ›” 17 16:55:08 imx327-desktop kernel:     GATHER at ffd50000+36b8, 2 words
[2022-05-17 16:55:12.762]  5μ›” 17 16:55:08 imx327-desktop kernel: 20000001 
[2022-05-17 16:55:12.762]  5μ›” 17 16:55:08 imx327-desktop kernel: 0000001f 
[2022-05-17 16:55:12.762]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.762]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.779]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.779]                                          ---- host general irq ----
[2022-05-17 16:55:12.779]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask_ext = 0xc0000000
[2022-05-17 16:55:12.795]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_hintmask = 0x80000000
[2022-05-17 16:55:12.796]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intc0mask = 0x00000001
[2022-05-17 16:55:12.796]  5μ›” 17 16:55:08 imx327-desktop kernel: sync_intmask = 0x00000011
[2022-05-17 16:55:12.813]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.813]                                          ---- host syncpt irq mask ----
[2022-05-17 16:55:12.813]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(0) = 0x54000001
[2022-05-17 16:55:12.829]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(1) = 0x44000001
[2022-05-17 16:55:12.829]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(2) = 0x00000010
[2022-05-17 16:55:12.829]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(3) = 0x00000000
[2022-05-17 16:55:12.846]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(4) = 0x00000000
[2022-05-17 16:55:12.846]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(5) = 0x00000000
[2022-05-17 16:55:12.863]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(6) = 0x00000000
[2022-05-17 16:55:12.863]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(7) = 0x00000000
[2022-05-17 16:55:12.879]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(8) = 0x00000000
[2022-05-17 16:55:12.879]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(9) = 0x00000000
[2022-05-17 16:55:12.879]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(10) = 0x00000000
[2022-05-17 16:55:12.896]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_int_mask(11) = 0x00000000
[2022-05-17 16:55:12.896]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.913]                                          ---- host syncpt irq status ----
[2022-05-17 16:55:12.913]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(0) = 0x00000000
[2022-05-17 16:55:12.913]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(1) = 0x00000000
[2022-05-17 16:55:12.929]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(2) = 0x00000000
[2022-05-17 16:55:12.929]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(3) = 0x00000000
[2022-05-17 16:55:12.946]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(4) = 0x00000000
[2022-05-17 16:55:12.946]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_thresh_cpu0_int_status(5) = 0x00000000
[2022-05-17 16:55:12.963]  5μ›” 17 16:55:08 imx327-desktop kernel: 
[2022-05-17 16:55:12.963]                                          ---- host syncpt thresh ----
[2022-05-17 16:55:12.963]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(0) = 1
[2022-05-17 16:55:12.978]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(13) = 36948742
[2022-05-17 16:55:12.978]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(14) = 36916610
[2022-05-17 16:55:12.994]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(15) = 113411996
[2022-05-17 16:55:12.998]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(16) = 36899989
[2022-05-17 16:55:12.998]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(29) = 36899984
[2022-05-17 16:55:13.015]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(31) = 73858635
[2022-05-17 16:55:13.015]  5μ›” 17 16:55:08 imx327-desktop kernel: syncpt_int_thresh_thresh_0(34) = 36899985
[2022-05-17 16:55:13.029]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: Error: waitCsiFrameStart timeout guid 2
[2022-05-17 16:55:13.029]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: ************VI/CSI Debug Registers**********
[2022-05-17 16:55:13.046]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CFG_INTERRUPT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.046]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CFG_INTERRUPT_STATUS_0 = 0x00000000
[2022-05-17 16:55:13.063]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CSI_0_ERROR_STATUS_0 = 0x00000000
[2022-05-17 16:55:13.063]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CSI_0_ERROR_INT_MASK_0 = 0x0000001f
[2022-05-17 16:55:13.079]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CSI_1_ERROR_STATUS_0 = 0x00000000
[2022-05-17 16:55:13.079]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: VI_CSI_1_ERROR_INT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.095]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_PIXEL_PARSER_A_INTERRUPT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.095]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_PIXEL_PARSER_A_STATUS_0 = 0x00000000
[2022-05-17 16:55:13.113]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_PIXEL_PARSER_B_INTERRUPT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.113]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_PIXEL_PARSER_B_STATUS_0 = 0x00000000
[2022-05-17 16:55:13.129]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CIL_A_INTERRUPT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.129]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CIL_A_STATUS_0 = 0x00000010
[2022-05-17 16:55:13.146]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CILA_STATUS_0 = 0x00040040
[2022-05-17 16:55:13.146]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CIL_B_INTERRUPT_MASK_0 = 0x00000000
[2022-05-17 16:55:13.162]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout: ISP port 0 timed out! (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 478)
[2022-05-17 16:55:13.179]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout:  (propagating from src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 524)
[2022-05-17 16:55:13.196]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 116)
[2022-05-17 16:55:13.196]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout: Worker thread IspHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)
[2022-05-17 16:55:13.213]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CIL_B_STATUS_0 = 0x00000010
[2022-05-17 16:55:13.230]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_CILB_STATUS_0 = 0x00040040
[2022-05-17 16:55:13.230]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_CSI_READONLY_STATUS_0 = 0x00000001
[2022-05-17 16:55:13.246]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_DEBUG_CONTROL_0 = 0x7e424b00
[2022-05-17 16:55:13.246]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_DEBUG_COUNTER_0_0 = 0x00000000
[2022-05-17 16:55:13.263]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_DEBUG_COUNTER_1_0 = 0x00000000
[2022-05-17 16:55:13.263]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: CSI_DEBUG_COUNTER_2_0 = 0x00000000
[2022-05-17 16:55:13.279]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: *****************************************
[2022-05-17 16:55:13.279]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: PowerServiceCore:handleRequests: timePassed = 1521
[2022-05-17 16:55:13.295]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout: ISP Stats timed out! (in src/services/capture/NvIspHw.cpp, function waitIspStatsFinished(), line 566)
[2022-05-17 16:55:13.313]  5μ›” 17 16:55:08 imx327-desktop nvargus-daemon[5347]: SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 997)
1 Like

The error tell capture failed from sensor and timeout. What’s the command for the use case?

We do not use the DeepStream SDK or the gstreamer command to view the IMX327 sensor.
We developed the IMX327 driver and encoder process based on the frontend samples.
What do you mean by saying β€œWhat’s the command for the use case?”

I mean how to reproduce the problem.
Does the problem always happen after 15 days?

Yes, this error occurs every 15 days.

Looks like memory leak cause the problem.

1 Like

You said " capture failed from sensor and timeout" as the cause of the error.
If you expect a memory leak, is it kernel space? Or is it user space?
In other words, where is the memory space used for capture?

We will do a test based on your comments.
We are going to test the memory state via the β€œfree” command.
Are there any other memory leak tests you would recommend?

Maybe can try restart nvargus-daemon after close camera.

sudo nvargus-daemon restart

Do you stream continuously without stream on/off to get the problem?

We didn’t power on/off or run the β€œsudo nvargus-daemon restart” command.
It seems that capture and compression are performed normally when the encoding process implemented based on the frontend is restarted.

3 days have passed since the restart of the encoding process and no memory leak is observed in monitoring through the β€œfree” command.

One question is, if there is a memory leak, OOM will work, but there is no record of OOM operation.

In addition, we implemented the encoding process that operates through the MIPI CSI yuv422 Zoom Camera and V4L2 (/dev/video0) interface, which has been compressing yuv data normally for 40 days until today.

The encoding process differs only in the path to obtain yuv data from libargus and /dev/video0.

That is, the encoding process is implemented to obtain yuv data from both libargus and /dev/video0, and H.264/H.265 is compressed through NVENC.