Problems with Video on Custom Board Rev.2 Fitted with ADV7282-M Chip

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
, ,
1

Hello,

We’ve been working on our custom board rev.2, which is equipped with an ADV7282-M chip configured via the adv7180.c driver. On the previous version of our board (rev.0), we had four ADV chips connected to CSI lanes 2, 3, 4, and 5, and everything worked perfectly. However, on our new rev.2 board, we only have one ADV chip connected to CSI lane 3, and we’re having some issues with the video playback.

The GStreamer pipeline we are using has been failing most of the time. It does work occasionally (about 5% of the time), but we haven’t been able to identify the conditions that make it succeed. Interestingly, the pipeline seems to function reliably, albeit at a very low framerate, when we access it over our office VPN.

Here’s a snippet of the debug output that we’ve identified as potentially relevant:

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not read from resource.
gstv4l2bufferpool.c(1054): gst_v4l2_buffer_pool_poll (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
poll error 1: Invalid argument (22)

We’ve also noticed some errors related to the Tegra VI and CSI subsystems in the dmesg output, including PXL_SOF syncpt timeout! err = -11, ATOMP_FE syncpt timeout!, and various csi4_stream_check_status errors with different INTR_STATUS and ERR_INTR_STATUS codes.

We’ve checked the physical connections and made sure the camera sensor is compatible with our system. We’ve also tried updating our system software and adjusting the settings of the CSI and VI subsystems. However, we’re still experiencing the same issues.

I’ve also attached parts of relevant schematics for the rev2 board.

We’re hoping that some of you might have encountered similar issues or could provide some insight into what might be going wrong. Any help would be greatly appreciated!

Other details:

  1. We are stuck on JP3.1 for this particular release.

Thank you in advance for your support.

Schematics
Schematics1115×518 69.5 KB

3

hello vK-embedded,

may I double confirm the Jetpack release version,
is this Jetpack-3.1/l4t-r28.1? this is one very old release version…

4

Yeah, I’m dealing with a bit of a pickle here. I’ve got a situation with r28.1, and the twist is I can’t really make changes to the JP version right now.

Now, I’ve run into this sort of snag with JP4.6 as well. But the fix I need this time around has to work for JP3.1. So that’s the challenge.

7

Hey Jerry, we could even figure out for JP4.6 and I’d personally see if that patch applies for JP3.1.

8

hello vK-embedded,

may I know what’s the patches.
the code-base is different. you might seeing conflicts if you apply JP-4.6 changes to JP-3.1 directly.

BTW,
you may see-also external wiki page for some known bug fixes and patches.
for example, Jetson/L4T/r28.2.1 patches - eLinux.org

9

Hey, Sorry about the delay. I have generated the patch files using diff.

They are for JP3.1
vi4_fops.txt (543 Bytes)
adv7180.txt (12.5 KB)

I hope they suffice.

Thanks!

10

hello vK-embedded,

you may ignore those conflicts, since it’s new added function, set_timestamp to update timestamp to vb2 buffer.
for example,

+void set_timestamp(struct tegra_channel_buffer *buf,
+                       const struct timespec *ts)
+{
+#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 9, 0)
+       /* update time stamp of the buffer */
+       buf->buf.timestamp.tv_sec = ts->tv_sec;
+       buf->buf.timestamp.tv_usec = ts->tv_nsec / NSEC_PER_USEC;
+#else
+       /* TODO: granular time code information */
+       buf->buf.timecode.seconds = ts->tv_sec;
+#endif

since this is upstream driver code, you may seeing git history to cherry-pick necessary changes to adapt your driver.

12

we are still facing the problem. then how it’s marked a solution? we have been able to capture video consistently using gstreamer pipeline but fps is very low. fps is 4.5 on average.

we tried your patches that you copy and paste on every post. please tell us something we don’t know. so that we can save some time.

v4l2-compliance is failing for this test.
fail: v4l2-test-io-config.cpp(174): TIMINGS cap not set, but could still get timings
fail: v4l2-test-io-config.cpp(239): Timings check failed for input 0.
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: FAIL

13

fps is very low with gstreamer. around 4.5. it’s showing 50 fps with v4l2-ctl.

this is the pipeline
gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw, width=720, height=576 ! videoconvert ! fpsdisplaysink text-overlay=true video-sink=ximagesink

FYI nvvidconv doesn’t work.

14

used nvidia plugins like nvvidconv with gstreamer pipeline. fps jumped to 7 from 4.5. impressive. but we want it to be atleast 30.

15

hello shivlal12345,

is there any failure when you running gst pipeline with v4l2src to test the stream?
BTW,
may I also know what’s the format dumps by… $ v4l2-ctl -d /dev/video0 --list-formats-ext
you may configure the pipeline correctly to avoid some internal process to lower the frame-rate.

16

v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2

v4l2-ctl --set-fmt-video=width=720,height=576,pixelformat=YUYV --stream-mmap --stream-count=100 -d /dev/video2
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.49 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

gstreamer pipeline
gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,height=576,width=720 ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=NV12’ ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=ximagesink sync=0 -v

17

debug log while capturing with gstreamer

kworker/0:1-9022 [000] …1 18284.858350: rtcpu_vinotify_handle_msg: tstamp:571656745803 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:426094874 data:0x00000001
kworker/0:1-9022 [000] …1 18284.858353: rtcpu_vinotify_handle_msg: tstamp:571656745962 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:426094905 data:0x00000000
kworker/0:1-9022 [000] …1 18284.910341: rtcpu_vinotify_handle_msg: tstamp:571657321669 tag:CHANSEL_PXL_EOF channel:0x00 frame:1 vi_tstamp:426670703 data:0x023f0002
kworker/0:1-9022 [000] …1 18284.910343: rtcpu_vinotify_handle_msg: tstamp:571657321773 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:426670750 data:0x00000000
kworker/0:1-9022 [000] …1 18284.910345: rtos_queue_peek_from_isr_failed: tstamp:571657988458 queue:0x0b4a3c58
kworker/0:1-9022 [000] …1 18285.014339: rtcpu_vinotify_handle_msg: tstamp:571661120867 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:430469907 data:0x00000001
kworker/0:1-9022 [000] …1 18285.014342: rtcpu_vinotify_handle_msg: tstamp:571661121016 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:430469937 data:0x00000000
kworker/0:1-9022 [000] …1 18285.014342: rtcpu_vinotify_handle_msg: tstamp:571661125260 tag:CHANSEL_LOAD_FRAMED channel:0x08 frame:2 vi_tstamp:430473498 data:0x08000000
kworker/0:1-9022 [000] …1 18285.014343: rtcpu_vinotify_handle_msg: tstamp:571661125360 tag:CHANSEL_FAULT_FE channel:0x08 frame:2 vi_tstamp:430473499 data:0x00000001
kworker/0:1-9022 [000] …1 18285.014344: rtcpu_vinotify_handle_msg: tstamp:571661125552 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:430473502 data:0x00000000
kworker/0:1-9022 [000] …1 18285.014345: rtos_queue_peek_from_isr_failed: tstamp:571661146081 queue:0x0b4a3c58
kworker/0:1-9022 [000] …1 18285.014346: rtos_queue_peek_from_isr_failed: tstamp:571661184408 queue:0x0b4a3c58
kworker/0:1-9022 [000] …1 18285.014347: rtcpu_start: tstamp:571661185098
kworker/0:1-9022 [000] …1 18285.014348: rtcpu_vinotify_handle_msg: tstamp:571661696584 tag:CSIMUX_FRAME channel:0x00 frame:2 vi_tstamp:431045778 data:0x00400063
kworker/0:1-9022 [000] …1 18285.014349: rtcpu_vinotify_handle_msg: tstamp:571661745844 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:431094911 data:0x00000001
kworker/0:1-9022 [000] …1 18285.014349: rtcpu_vinotify_handle_msg: tstamp:571661745990 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:431094941 data:0x00000000
kworker/0:1-9022 [000] …1 18285.014350: rtcpu_vinotify_handle_msg: tstamp:571661749898 tag:CHANSEL_LOAD_FRAMED channel:0x08 frame:1 vi_tstamp:431098156 data:0x08000000
kworker/0:1-9022 [000] …1 18285.014350: rtcpu_vinotify_handle_msg: tstamp:571661749999 tag:CHANSEL_FAULT_FE channel:0x08 frame:1 vi_tstamp:431098161 data:0x00000001
kworker/0:1-9022 [000] …1 18285.014351: rtcpu_vinotify_handle_msg: tstamp:571661750193 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:431098165 data:0x00000000
kworker/0:1-9022 [000] …1 18285.014352: rtos_queue_peek_from_isr_failed: tstamp:571661770714 queue:0x0b4a3c58

18

dmesg log. but the following errors are occasional occurrences.

[18285.002454] No vi channel is active
[18285.002626] nvcsi 150c0000.nvcsi: csi4_stream_check_status (3) INTR_STATUS 0x00000004
[18285.002634] nvcsi 150c0000.nvcsi: csi4_stream_check_status (3) ERR_INTR_STATUS 0x00000004

[47674.417659] tegra-vi4 15700000.vi: Status: 7 channel:00 frame:0002
[47674.417667] tegra-vi4 15700000.vi: timestamp sof 47682573408832 eof 47682591776576 data 0x00000001
[47674.417673] tegra-vi4 15700000.vi: capture_id 101342 stream 3 vchan 0
[47675.397965] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[47676.397967] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[47676.398183] nvcsi 150c0000.nvcsi: csi4_stream_check_status (3) INTR_STATUS 0x00000008
[47676.398191] nvcsi 150c0000.nvcsi: csi4_stream_check_status (3) ERR_INTR_STATUS 0x00000008

19

hello shivlal12345,

you may exclude video converter,
please give it a try to fetch the stream as following.
$ gst-launch-1.0 v4l2src device=/dev/video2 ! "video/x-raw, format=(string)YUYV, width=(int)720, height=(int)576" ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v -e

your VI tracing logs show frame packet has sending to VI engine correctly.
it’s PXL_SOF syncpt timeout! which means it sometime cannot wait for start-of-frame.
may I know what’s the repo step in details? for example, did you disconnect the camera device, or there’s intermittently MIPI signal sending to VI engine?

besides,
please also try to configure system clocks with below commands to boost all the VI/CSI/ISP clocks.
for example,

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
20

Hi JerryChang,

I am getting an error without videoconvert.
gst-launch-1.0 v4l2src device=/dev/video2 ! “video/x-raw, format=(string)YUYV, width=(int)720, height=(int)576” ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v -e
WARNING: erroneous pipeline: could not link v4l2src0 to fpsdisplaysink0

we are not disconnecting the camera. its connected all the time. although there is a loose connection sometimes.

we already tried boosting the clocks. no change.

21

anything we can do to increase fps with gstreamer? it’s very urgent. you timely help would be appreciated.

22

I’ve check this command works normally from my side.
for example,

$ gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v -e
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 61, dropped: 0, current: 59.95, average: 60.97
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 91, dropped: 0, current: 59.92, average: 60.62
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 121, dropped: 0, current: 59.93, average: 60.45
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 151, dropped: 0, current: 59.87, average: 60.33

the failure you seen… WARNING: erroneous pipeline: could not link v4l2src0 to fpsdisplaysink0 should be the format mismatch issue.
are you sure sensor is sending YUYV actually?

23

how do we confirm the format from camera? its supposed to be YUYV. tried UYVY as well.

24

we have switched to jetpack 4.5. not having this issue anymore.