Screen-flicker when capturing images from image sensor

Me and my team use a system including a Jetson TX2 and an external image sensor. For a long time we have been running L4T 28.1 with a custom image sensor driver based directly on the ov5693.c example driver. We interface to the image sensor using iotctl and the V4L2 API. This system works very well.

We want to upgrade to L4T 28.2 to enable TensorRT. We recently upgraded the kernel on a device which also entailed updating the driver to match the ov5693.c that comes with the 28.2 packages. When we now try to capture images and display them, the entire screen flickers in random patches (even outside of the window holding the displayed image). Underneath all the flickering we can see that the image sensor is still capturing images.

The output of the tracer is included below. The error we have focused on is

arm_smmu_context_fault: Unhandled context fault: iova=0x61d88000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=ea965003, pte=0

On the embedded Linux wiki, it was suggested that this error arises when the driver’s embedded_metadata_height is set to 0 or does not match the sensor’s output. We matched what we found in the data sheet but just to be sure, we cycled through several different embedded_metadata_height values and the problem still occurs.

Any help would be greatly appreciated. This is a significant blocker for us right now. Trace output below (shortened a bit, there was a lot of the same thing repeated since we are live streaming):

# entries-in-buffer/entries-written: 1385/1385   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/0:1-57    [000] ...1   273.727309: rtos_queue_peek_from_isr_failed: tstamp:8883737809 queue:0x0b4a3c58
     kworker/0:1-57    [000] ...1   273.727313: rtcpu_start: tstamp:8883738767
          <idle>-0     [000] dnh1   273.829001: arm_smmu_context_fault: Unhandled context fault: iova=0x602d0000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
     kworker/0:1-57    [000] ...1   273.831330: rtcpu_vinotify_handle_msg: tstamp:8886794397 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:296859254 data:0x00000001
     kworker/0:1-57    [000] ...1   273.831333: rtcpu_vinotify_handle_msg: tstamp:8886794562 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:296859258 data:0x00000000
     kworker/0:1-57    [000] ...1   273.831334: rtcpu_vinotify_handle_msg: tstamp:8886796867 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:2 vi_tstamp:296861861 data:0x08000000
          <idle>-0     [000] dnh1   273.857273: arm_smmu_context_fault: Unhandled context fault: iova=0x61d88000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=ea965003, pte=0
            Xorg-860   [000] d.h1   273.889614: arm_smmu_context_fault: Unhandled context fault: iova=0x636c8000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=a7b14003, pte=0
     kworker/0:1-57    [000] ...1   273.889732: rtcpu_vinotify_handle_msg: tstamp:8887825921 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:297890740 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   273.889734: rtcpu_vinotify_handle_msg: tstamp:8887826032 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:297890757 data:0x00000000
     kworker/0:1-57    [000] ...1   273.889735: rtcpu_vinotify_handle_msg: tstamp:8887835822 tag:CHANSEL_PXL_SOF channel:0x00 frame:3 vi_tstamp:297900692 data:0x00000001
     kworker/0:1-57    [000] ...1   273.889736: rtcpu_vinotify_handle_msg: tstamp:8887835967 tag:ATOMP_FS channel:0x00 frame:3 vi_tstamp:297900696 data:0x00000000
     kworker/0:1-57    [000] ...1   273.889737: rtcpu_vinotify_handle_msg: tstamp:8887838695 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:3 vi_tstamp:297903689 data:0x08000000
     kworker/0:1-57    [000] ...1   273.889739: rtos_queue_peek_from_isr_failed: tstamp:8888738661 queue:0x0b4a3c58
     kworker/0:1-57    [000] ...1   273.889740: rtcpu_vinotify_handle_msg: tstamp:8888867349 tag:CHANSEL_PXL_EOF channel:0x00 frame:3 vi_tstamp:298932179 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   273.889740: rtcpu_vinotify_handle_msg: tstamp:8888867459 tag:ATOMP_FE channel:0x00 frame:3 vi_tstamp:298932196 data:0x00000000
     kworker/0:1-57    [000] ...1   273.889741: rtcpu_vinotify_handle_msg: tstamp:8888877263 tag:CHANSEL_PXL_SOF channel:0x00 frame:4 vi_tstamp:298942131 data:0x00000001
     kworker/0:1-57    [000] ...1   273.889742: rtcpu_vinotify_handle_msg: tstamp:8888877410 tag:ATOMP_FS channel:0x00 frame:4 vi_tstamp:298942135 data:0x00000000
     kworker/0:1-57    [000] ...1   273.889743: rtcpu_vinotify_handle_msg: tstamp:8888879295 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:4 vi_tstamp:298944291 data:0x08000000
      cbc_gui_qt-2714  [000] d.h1   273.927639: arm_smmu_context_fault: Unhandled context fault: iova=0x65448000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16b814003, pte=0
      cbc_gui_qt-2714  [000] dnH1   273.942469: arm_smmu_context_fault: Unhandled context fault: iova=0x65f90280, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16b80f003, pte=0
     kworker/0:1-57    [000] ...1   273.943106: rtcpu_vinotify_handle_msg: tstamp:8889908807 tag:CHANSEL_PXL_EOF channel:0x00 frame:4 vi_tstamp:299973617 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   273.943109: rtcpu_vinotify_handle_msg: tstamp:8889908917 tag:ATOMP_FE channel:0x00 frame:4 vi_tstamp:299973635 data:0x00000000
     kworker/0:1-57    [000] ...1   273.943110: rtcpu_vinotify_handle_msg: tstamp:8889918698 tag:CHANSEL_PXL_SOF channel:0x00 frame:5 vi_tstamp:299983570 data:0x00000001
     kworker/0:1-57    [000] ...1   273.943110: rtcpu_vinotify_handle_msg: tstamp:8889918842 tag:ATOMP_FS channel:0x00 frame:5 vi_tstamp:299983574 data:0x00000000
     kworker/0:1-57    [000] ...1   273.943111: rtcpu_vinotify_handle_msg: tstamp:8889920695 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:5 vi_tstamp:299985689 data:0x08000000
     kworker/0:1-57    [000] ...1   273.943112: rtcpu_vinotify_handle_msg: tstamp:8890950230 tag:CHANSEL_PXL_EOF channel:0x00 frame:5 vi_tstamp:301015057 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   273.943113: rtcpu_vinotify_handle_msg: tstamp:8890950335 tag:ATOMP_FE channel:0x00 frame:5 vi_tstamp:301015073 data:0x00000000
     kworker/0:1-57    [000] ...1   273.943113: rtcpu_vinotify_handle_msg: tstamp:8890960136 tag:CHANSEL_PXL_SOF channel:0x00 frame:6 vi_tstamp:301025009 data:0x00000001
     kworker/0:1-57    [000] ...1   273.943114: rtcpu_vinotify_handle_msg: tstamp:8890960279 tag:ATOMP_FS channel:0x00 frame:6 vi_tstamp:301025013 data:0x00000000
     kworker/0:1-57    [000] ...1   273.943115: rtcpu_vinotify_handle_msg: tstamp:8891065738 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:6 vi_tstamp:301130733 data:0x08000000
          <idle>-0     [000] dnh1   273.977160: arm_smmu_context_fault: Unhandled context fault: iova=0x67aa0000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=2464a8003, pte=0
          compiz-1544  [000] d.h.   274.003837: arm_smmu_context_fault: Unhandled context fault: iova=0x68f90000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=24b883003, pte=0
     kworker/0:1-57    [000] ...1   274.004459: rtcpu_vinotify_handle_msg: tstamp:8891991674 tag:CHANSEL_PXL_EOF channel:0x00 frame:6 vi_tstamp:302056495 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   274.004463: rtcpu_vinotify_handle_msg: tstamp:8891991788 tag:ATOMP_FE channel:0x00 frame:6 vi_tstamp:302056513 data:0x00000000
     kworker/0:1-57    [000] ...1   274.004463: rtcpu_vinotify_handle_msg: tstamp:8892001574 tag:CHANSEL_PXL_SOF channel:0x00 frame:7 vi_tstamp:302066447 data:0x00000001
     kworker/0:1-57    [000] ...1   274.004464: rtcpu_vinotify_handle_msg: tstamp:8892001717 tag:ATOMP_FS channel:0x00 frame:7 vi_tstamp:302066451 data:0x00000000
     kworker/0:1-57    [000] ...1   274.004465: rtcpu_vinotify_handle_msg: tstamp:8892149979 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:7 vi_tstamp:302214975 data:0x08000000
   kworker/u12:0-6     [000] d.h1   274.029778: arm_smmu_context_fault: Unhandled context fault: iova=0x6a3f0000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=243cc8003, pte=0
          compiz-1544  [000] d.h.   274.059658: arm_smmu_context_fault: Unhandled context fault: iova=0x6bb50000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=1db721003, pte=0
     kworker/0:1-57    [000] ...1   274.060252: rtcpu_vinotify_handle_msg: tstamp:8893033110 tag:CHANSEL_PXL_EOF channel:0x00 frame:7 vi_tstamp:303097934 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   274.060255: rtcpu_vinotify_handle_msg: tstamp:8893033220 tag:ATOMP_FE channel:0x00 frame:7 vi_tstamp:303097951 data:0x00000000
     kworker/0:1-57    [000] ...1   274.060255: rtcpu_vinotify_handle_msg: tstamp:8893043017 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:303107886 data:0x00000001
     kworker/0:1-57    [000] ...1   274.060256: rtcpu_vinotify_handle_msg: tstamp:8893043163 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:303107890 data:0x00000000
     kworker/0:1-57    [000] ...1   274.060257: rtcpu_vinotify_handle_msg: tstamp:8893045763 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:8 vi_tstamp:303110756 data:0x08000000
     kworker/0:1-57    [000] ...1   274.060259: rtos_queue_peek_from_isr_failed: tstamp:8893739171 queue:0x0b4a3c58
     kworker/0:1-57    [000] ...1   274.060260: rtcpu_vinotify_handle_msg: tstamp:8894074556 tag:CHANSEL_PXL_EOF channel:0x00 frame:8 vi_tstamp:304139373 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   274.060260: rtcpu_vinotify_handle_msg: tstamp:8894074663 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:304139390 data:0x00000000
     kworker/0:1-57    [000] ...1   274.060261: rtcpu_vinotify_handle_msg: tstamp:8894084451 tag:CHANSEL_PXL_SOF channel:0x00 frame:9 vi_tstamp:304149325 data:0x00000001
     kworker/0:1-57    [000] ...1   274.060262: rtcpu_vinotify_handle_msg: tstamp:8894084595 tag:ATOMP_FS channel:0x00 frame:9 vi_tstamp:304149329 data:0x00000000
     kworker/0:1-57    [000] ...1   274.060263: rtcpu_vinotify_handle_msg: tstamp:8894086715 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:9 vi_tstamp:304151715 data:0x08000000
          compiz-1544  [000] d.h.   274.089864: arm_smmu_context_fault: Unhandled context fault: iova=0x6d2f0000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16b84e003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108300: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecd8000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108329: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecde800, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108356: arm_smmu_context_fault: Unhandled context fault: iova=0x6ece3e80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108383: arm_smmu_context_fault: Unhandled context fault: iova=0x6ece9000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108409: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecee4c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108436: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecf3640, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108463: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecf8c40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108489: arm_smmu_context_fault: Unhandled context fault: iova=0x6ecfdd40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108516: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed03200, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108542: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed08280, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108569: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed0d600, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108595: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed12780, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108621: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed17b80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108648: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed1cc80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108675: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed22080, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108701: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed27200, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108727: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed2c5c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108755: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed31800, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108781: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed36f80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108808: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed3bec0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108834: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed41400, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108863: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed46280, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108889: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed4be80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108915: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed51000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108943: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed56200, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108969: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed5b7c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.108995: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed60700, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109020: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed65900, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109046: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed6ab00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109072: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed6f980, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109098: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed74c40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109124: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed79ac0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109150: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed7ec80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109176: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed83b40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109202: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed88ec0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109230: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed8dd00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109257: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed93840, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109283: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed98b00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109309: arm_smmu_context_fault: Unhandled context fault: iova=0x6ed9de00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109335: arm_smmu_context_fault: Unhandled context fault: iova=0x6eda2c80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109361: arm_smmu_context_fault: Unhandled context fault: iova=0x6eda7f00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109387: arm_smmu_context_fault: Unhandled context fault: iova=0x6edacd80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109412: arm_smmu_context_fault: Unhandled context fault: iova=0x6edb2040, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109441: arm_smmu_context_fault: Unhandled context fault: iova=0x6edb7100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109466: arm_smmu_context_fault: Unhandled context fault: iova=0x6edbc8c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109492: arm_smmu_context_fault: Unhandled context fault: iova=0x6edc1780, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109518: arm_smmu_context_fault: Unhandled context fault: iova=0x6edc6a00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109546: arm_smmu_context_fault: Unhandled context fault: iova=0x6edcbb00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109572: arm_smmu_context_fault: Unhandled context fault: iova=0x6edd1100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109600: arm_smmu_context_fault: Unhandled context fault: iova=0x6edd6000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109625: arm_smmu_context_fault: Unhandled context fault: iova=0x6eddb900, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109651: arm_smmu_context_fault: Unhandled context fault: iova=0x6ede0780, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109677: arm_smmu_context_fault: Unhandled context fault: iova=0x6ede5940, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h1   274.109704: arm_smmu_context_fault: Unhandled context fault: iova=0x6edeaa00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h1   274.109730: arm_smmu_context_fault: Unhandled context fault: iova=0x6edefe00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h1   274.109758: arm_smmu_context_fault: Unhandled context fault: iova=0x6edf4f00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h1   274.109784: arm_smmu_context_fault: Unhandled context fault: iova=0x6edfa640, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h1   274.109810: arm_smmu_context_fault: Unhandled context fault: iova=0x6edff580, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df80003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109836: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee047c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109862: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee09900, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109888: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee0eb80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109916: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee13e00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109942: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee19500, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109968: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee1e340, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.109994: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee23500, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110022: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee28800, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110048: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee2df00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110074: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee32d80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110100: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee38040, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110128: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee3d200, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110154: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee42a00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110180: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee47800, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110206: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee4ca00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110234: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee51c00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110259: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee57200, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110285: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee5c100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110312: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee61400, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110339: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee66600, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110365: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee6bac0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110391: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee70b00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110417: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee75e40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110445: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee7b000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110471: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee80600, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110497: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee85500, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110523: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee8a700, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110549: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee8f9c0, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110574: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee94800, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110600: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee99a40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110626: arm_smmu_context_fault: Unhandled context fault: iova=0x6ee9e940, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110652: arm_smmu_context_fault: Unhandled context fault: iova=0x6eea3b40, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110679: arm_smmu_context_fault: Unhandled context fault: iova=0x6eea8a80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110705: arm_smmu_context_fault: Unhandled context fault: iova=0x6eeadf00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110731: arm_smmu_context_fault: Unhandled context fault: iova=0x6eeb2e00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110756: arm_smmu_context_fault: Unhandled context fault: iova=0x6eeb8040, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110785: arm_smmu_context_fault: Unhandled context fault: iova=0x6eebd000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110810: arm_smmu_context_fault: Unhandled context fault: iova=0x6eec2980, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110836: arm_smmu_context_fault: Unhandled context fault: iova=0x6eec7700, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110862: arm_smmu_context_fault: Unhandled context fault: iova=0x6eecca00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110889: arm_smmu_context_fault: Unhandled context fault: iova=0x6eed1a00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110915: arm_smmu_context_fault: Unhandled context fault: iova=0x6eed6f80, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110941: arm_smmu_context_fault: Unhandled context fault: iova=0x6eedbf00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=0
 nvgpu_channel_p-892   [000] d.h.   274.110967: arm_smmu_context_fault: Unhandled context fault: iova=0x6eee1100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=16df81003, pte=60000224e8cf43
     kworker/0:1-57    [000] ...1   274.111330: rtcpu_vinotify_handle_msg: tstamp:8895115992 tag:CHANSEL_PXL_EOF channel:0x00 frame:9 vi_tstamp:305180812 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   274.111332: rtcpu_vinotify_handle_msg: tstamp:8895116102 tag:ATOMP_FE channel:0x00 frame:9 vi_tstamp:305180829 data:0x00000000
     kworker/0:1-57    [000] ...1   274.111333: rtcpu_vinotify_handle_msg: tstamp:8895125897 tag:CHANSEL_PXL_SOF channel:0x00 frame:10 vi_tstamp:305190763 data:0x00000001
     kworker/0:1-57    [000] ...1   274.111334: rtcpu_vinotify_handle_msg: tstamp:8895126041 tag:ATOMP_FS channel:0x00 frame:10 vi_tstamp:305190768 data:0x00000000
     kworker/0:1-57    [000] ...1   274.111334: rtcpu_vinotify_handle_msg: tstamp:8895128081 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:10 vi_tstamp:305193078 data:0x08000000
     kworker/0:1-57    [000] ...1   274.111335: rtcpu_vinotify_handle_msg: tstamp:8896157430 tag:CHANSEL_PXL_EOF channel:0x00 frame:10 vi_tstamp:306222251 data:0x0c2f0002
     kworker/0:1-57    [000] ...1   274.111336: rtcpu_vinotify_handle_msg: tstamp:8896157547 tag:ATOMP_FE channel:0x00 frame:10 vi_tstamp:306222268 data:0x00000000
     kworker/0:1-57    [000] ...1   274.111337: rtcpu_vinotify_handle_msg: tstamp:8896167332 tag:CHANSEL_PXL_SOF channel:0x00 frame:11 vi_tstamp:306232203 data:0x00000001
     kworker/0:1-57    [000] ...1   274.111337: rtcpu_vinotify_handle_msg: tstamp:8896167476 tag:ATOMP_FS channel:0x00 frame:11 vi_tstamp:306232207 data:0x00000000
     kworker/0:1-57    [000] ...1   274.111338: rtcpu_vinotify_handle_msg: tstamp:8896169304 tag:CHANSEL_LOAD_FRAMED channel:0x10 frame:11 vi_tstamp:306234302 data:0x08000000
          <idle>-0     [000] dnh1   274.141668: arm_smmu_context_fault: Unhandled context fault: iova=0x602e0000, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141699: arm_smmu_context_fault: Unhandled context fault: iova=0x602e6c00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141726: arm_smmu_context_fault: Unhandled context fault: iova=0x602ec600, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141753: arm_smmu_context_fault: Unhandled context fault: iova=0x602f1900, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141781: arm_smmu_context_fault: Unhandled context fault: iova=0x602f7100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141809: arm_smmu_context_fault: Unhandled context fault: iova=0x602fc500, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141836: arm_smmu_context_fault: Unhandled context fault: iova=0x60301b00, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0
          <idle>-0     [000] dnh1   274.141863: arm_smmu_context_fault: Unhandled context fault: iova=0x60307100, fsynr=0x13, cb=19, sid=4(0x4 - VI), pgd=26affe003 pud=26affe003, pmd=d54f9003, pte=0