V4L2 issues in userpointer method implementation

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,

I’m trying to implement the V4L2 user pointer method. I’m using NVIDIA XAVIER NX.

Problem: When I’m updating the “buf.m.userptr” on each deque sometimes I get
"
tegra194-vi5 15c10000.vi: vi_capture_status : no reply from camera processor
tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
"
Also, I printed the errorno just after the dequeue. It says “Inappropriate ioctl for device”

But when I don’t update the “buf.m.userptr” on each deque then it works fine.

Can someone suggest what is going wrong while updating the “buf.m.userptr”?

4

Have a reference to this topic to check by v4l2-ctl-t186 and then reference to the MMAPI sample code.

5

Hi Shane,

Thank you for your response.

We are still facing the same issue. We tried using “v4l2-ctl-t186.txt”. We’re getting :

VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_ENUM_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
Width/Height : 1248/2048
Pixel Format : ‘RG10’
Field : None
Bytes per Line : 2496
Size Image : 5111808
Colorspace : sRGB
Transfer Function : Default
YCbCr Encoding : Default
Quantization : Default
Flags :
Max Framerate : 0
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
Index : 0
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 0
Length : 5111808
Bytesused: 5111808
Timestamp: 0.000000s (Monotonic, End-of-Frame)

6

What’s your command line? Does mmap working for your?

v4l2-ctl --stream-mmap --stream-count=1 -d /dev/video0 --verbose
7

We used:
./v4l2-ctl-t186 --set-fmt-video=width=1248,height=2048,pixelformat=SBGGR10 --stream-mmap --stream-count=32 --stream-to=/media/sdcard/video2.raw -d /dev/video0 --verbose

8

Please confirm --stream-mmap working or not?

9

It’s not working.

10

OK, that’s tell your problem don’t matter with the userpointer.
Have a reference to below link to check the trace log for sensor bring up.

https://elinux.org/Jetson/l4t/Camera_BringUp

11

Hi Shane,

Thanks for this doc. Finally, we have made MMAP working.
Now, we are facing issues with USER_PTR method.
v4l2-ctl command: strace -yy -tt -T v4l2-ctl --set-fmt-video=width=1472,height=1080 --stream-user -d /dev/video2 --verbose
Below Is the issue (with strace) we are facing:

02:14:23.460531 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b492c8) = 0 <0.000044>
02:14:23.460734 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000035>
02:14:23.460893 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000033>
02:14:23.461090 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000036>
02:14:23.461271 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000054>
02:14:23.461525 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000053>
02:14:23.461791 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000053>
02:14:23.462266 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000061>
02:14:23.462661 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000057>
02:14:23.462937 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000048>
02:14:23.463210 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000050>
02:14:23.463452 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000151>
02:14:23.463775 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000058>
02:14:23.464026 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000050>
02:14:23.464287 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000049>
02:14:23.464591 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000055>
02:14:23.464848 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000059>
02:14:23.465097 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000040>
02:14:23.465309 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000038>
02:14:23.465541 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000036>
02:14:23.465776 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000039>
02:14:23.466032 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = 0 <0.000058>
02:14:23.466289 ioctl(3</dev/video2>, VIDIOC_QUERY_EXT_CTRL, 0x7fe1b493b0) = -1 EINVAL (Invalid argument) <0.000050>
02:14:23.466556 ioctl(3</dev/video2>, VIDIOC_G_FMT, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, fmt.pix={width=1472, height=1080, pixelformat=v4l2_fourcc(‘R’, ‘G’, ‘1’, ‘0’), field=V4L2_FIELD_NONE, bytesperline=2944, sizeimage=3179520, colorspace=V4L2_COLORSPACE_SRGB}}) = 0 <0.000050>
02:14:23.466838 write(1</dev/pts/0>, “VIDIOC_G_FMT: ok\n”, 17VIDIOC_G_FMT: ok
) = 17 <0.000116>
02:14:23.467140 ioctl(3</dev/video2>, VIDIOC_S_FMT, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, fmt.pix={width=1472, height=1080, pixelformat=v4l2_fourcc(‘R’, ‘G’, ‘1’, ‘0’), field=V4L2_FIELD_NONE, bytesperline=0, sizeimage=3179520, colorspace=V4L2_COLORSPACE_SRGB} => fmt.pix={width=1472, height=1080, pixelformat=v4l2_fourcc(‘R’, ‘G’, ‘1’, ‘0’), field=V4L2_FIELD_NONE, bytesperline=2944, sizeimage=3179520, colorspace=V4L2_COLORSPACE_SRGB}}) = 0 <0.000084>
02:14:23.467437 write(1</dev/pts/0>, “VIDIOC_S_FMT: ok\n”, 17VIDIOC_S_FMT: ok
) = 17 <0.000125>
02:14:23.467737 write(1</dev/pts/0>, “Format Video Capture:\n”, 22Format Video Capture:
) = 22 <0.000100>
02:14:23.468008 write(1</dev/pts/0>, “\tWidth/Height : 1472/1080\n”, 31 Width/Height : 1472/1080
) = 31 <0.000110>
02:14:23.468270 write(1</dev/pts/0>, “\tPixel Format : ‘RG10’\n”, 28 Pixel Format : ‘RG10’
) = 28 <0.000146>
02:14:23.468604 write(1</dev/pts/0>, “\tField : None\n”, 26 Field : None
) = 26 <0.000089>
02:14:23.468887 write(1</dev/pts/0>, “\tBytes per Line : 2944\n”, 26 Bytes per Line : 2944
) = 26 <0.000118>
02:14:23.469243 write(1</dev/pts/0>, “\tSize Image : 3179520\n”, 29 Size Image : 3179520
) = 29 <0.000124>
02:14:23.469565 write(1</dev/pts/0>, “\tColorspace : sRGB\n”, 26 Colorspace : sRGB
) = 26 <0.000084>
02:14:23.469853 write(1</dev/pts/0>, “\tTransfer Function : Default (ma”…, 44 Transfer Function : Default (maps to sRGB)
) = 44 <0.000157>
02:14:23.470206 write(1</dev/pts/0>, “\tYCbCr/HSV Encoding: Default (ma”…, 49 YCbCr/HSV Encoding: Default (maps to ITU-R 601)
) = 49 <0.000105>
02:14:23.470504 write(1</dev/pts/0>, “\tQuantization : Default (ma”…, 50 Quantization : Default (maps to Full Range)
) = 50 <0.000114>
02:14:23.470768 write(1</dev/pts/0>, “\tFlags : \n”, 22 Flags :
) = 22 <0.000094>
02:14:23.471016 fcntl(3</dev/video2>, F_GETFL) = 0x20002 (flags O_RDWR|O_LARGEFILE) <0.000056>
02:14:23.471209 ioctl(3</dev/video2>, VIDIOC_SUBSCRIBE_EVENT, 0x7fe1b444f0) = -1 EINVAL (Invalid argument) <0.000062>
02:14:23.471415 ioctl(3</dev/video2>, VIDIOC_G_INPUT, [0]) = 0 <0.000238>
02:14:23.471797 ioctl(3</dev/video2>, VIDIOC_ENUMINPUT, {index=0, name=“Camera 4”, type=V4L2_INPUT_TYPE_CAMERA}) = 0 <0.000104>
02:14:23.472085 ioctl(3</dev/video2>, VIDIOC_REQBUFS, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, memory=V4L2_MEMORY_USERPTR, count=4 => 4}) = 0 <0.000112>
02:14:23.472390 write(1</dev/pts/0>, “VIDIOC_REQBUFS: ok\n”, 19VIDIOC_REQBUFS: ok
) = 19 <0.000119>
02:14:23.472658 ioctl(3</dev/video2>, VIDIOC_QUERYBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, index=0, memory=V4L2_MEMORY_USERPTR, m.userptr=NULL, length=3179520, bytesused=0, flags=V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC, …}) = 0 <0.000081>
02:14:23.472915 write(1</dev/pts/0>, “VIDIOC_QUERYBUF: ok\n”, 20VIDIOC_QUERYBUF: ok
) = 20 <0.000089>
02:14:23.473175 mmap(NULL, 3182592, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f96ae3000 <0.000111>
02:14:23.473471 ioctl(3</dev/video2>, VIDIOC_QBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, index=0}) = -1 EINVAL (Invalid argument) <0.000086>
02:14:23.473707 write(1</dev/pts/0>, “VIDIOC_QBUF: failed: Invalid arg”…, 38VIDIOC_QBUF: failed: Invalid argument
) = 38 <0.000101>
02:14:23.473991 close(3</dev/video2>) = 0 <0.001160>
02:14:23.475446 exit_group(-1) = ?
02:14:23.476022 +++ exited with 255 +++

12

Did you use v4l2-ctl-t186 for --stream-user?

13

Hi, It is working with the “v4l2-ctl-t186”. But I’m not sure how can this help to resolve the actual issue which I face when I’m updating the buf.m.userptr on each dequeue. Can you please help?

14

As my comment early to use MMAPI sample and compare your code with it to figure the problem.

15

Hi I checked the flow. It seems all correct. Then I enabled more debug logs. Here is the failure that I got

 kworker/0:3-11267 [000] ....  1494.144188: rtos_queue_send_from_isr_failed: tstamp:47080504980 queue:0x0bcb41f8
 kworker/0:3-11267 [000] ....  1494.144188: rtos_queue_send_from_isr_failed: tstamp:47080505135 queue:0x0bcb8a60
 kworker/0:3-11267 [000] ....  1494.144188: rtos_queue_send_from_isr_failed: tstamp:47080505291 queue:0x0bcba5e0
 kworker/0:3-11267 [000] ....  1494.144189: rtos_queue_send_from_isr_failed: tstamp:47080505436 queue:0x0bcbb3a0
 kworker/0:3-11267 [000] ....  1494.144189: rtos_queue_send_from_isr_failed: tstamp:47080505581 queue:0x0bcbc160
 kworker/0:3-11267 [000] ....  1494.144190: rtos_queue_send_from_isr_failed: tstamp:47080523378 queue:0x0bcb41f8
 kworker/0:3-11267 [000] ....  1494.144190: rtos_queue_send_from_isr_failed: tstamp:47080523549 queue:0x0bcb8a60
 kworker/0:3-11267 [000] ....  1494.144190: rtos_queue_send_from_isr_failed: tstamp:47080523704 queue:0x0bcba5e0
 kworker/0:3-11267 [000] ....  1494.144191: rtos_queue_send_from_isr_failed: tstamp:47080523854 queue:0x0bcbb3a0
 kworker/0:3-11267 [000] ....  1494.144191: rtos_queue_send_from_isr_failed: tstamp:47080524003 queue:0x0bcbc160
 kworker/0:3-11267 [000] ....  1494.144191: rtos_queue_send_from_isr_failed: tstamp:47080613828 queue:0x0bcb41f8
 kworker/0:3-11267 [000] ....  1494.144192: rtos_queue_send_from_isr_failed: tstamp:47080613979 queue:0x0bcb8a60
 kworker/0:3-11267 [000] ....  1494.144192: rtos_queue_send_from_isr_failed: tstamp:47080614132 queue:0x0bcba5e0
 kworker/0:3-11267 [000] ....  1494.144192: rtos_queue_send_from_isr_failed: tstamp:47080614278 queue:0x0bcbb3a0
 kworker/0:3-11267 [000] ....  1494.144193: rtos_queue_send_from_isr_failed: tstamp:47080614422 queue:0x0bcbc160
 kworker/0:3-11267 [000] ....  1494.144207: rtcpu_nvcsi_intr: tstamp:47080619094 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000110
 kworker/0:3-11267 [000] ....  1494.144216: rtcpu_nvcsi_intr: tstamp:47080619094 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000110
 kworker/0:3-11267 [000] ....  1494.144223: rtcpu_nvcsi_intr: tstamp:47080619094 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000010
 kworker/0:3-11267 [000] ....  1494.144231: rtcpu_nvcsi_intr: tstamp:47080619822 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x00000010
 kworker/0:3-11267 [000] ....  1494.144238: rtcpu_nvcsi_intr: tstamp:47080620292 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000110
 kworker/0:3-11267 [000] ....  1494.144245: rtcpu_nvcsi_intr: tstamp:47080620292 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x00000010
 kworker/0:3-11267 [000] ....  1494.144259: rtcpu_nvcsi_intr: tstamp:47080620292 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
 kworker/0:3-11267 [000] ....  1494.144267: rtcpu_nvcsi_intr: tstamp:47080620292 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000110
 kworker/0:3-11267 [000] ....  1494.144274: rtcpu_nvcsi_intr: tstamp:47080620292 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000010
16

Try boost the clocks to try.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
17

I have done it already. After doing this only I got that failure in trace.

18

I think could you the sensor problem doesn’t matter with the userptr.
Do you have any other sensor module to confirm it.

19

I tried with different sensors also and the same issue is coming.
Just to give you a little more background. I’m using 3 cameras simultaneously. For each camera, I’m storing around 1 sec of data. That is why I’m updating the buf.m.userptr. For few seconds everything works fine. But after some time the issue arise. It is very random and any camera can throw it. But the issue occurs every time.

tegra194-vi5 15c10000.vi: vi_capture_status : no reply from camera processor
tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms

Just after Dqueue

STRERR: Inappropriate ioctl for device

20

It could be the pipeline stuck cause the problem.
Please have a try to increase the QUEUED_BUFFERS to try.

…/kernel/nvidia/include/media/mc_common.h

21

I tried it with different buffer sizes of 8, 16 & 32.
Nothing is helping.

22

Does the same configure without problem for --stream-mmap?