RP IMX477 Camera shows up as /dev/video0 but reading data hangs

I have successfully gotten my Xavier NX Devkit to read video from a Raspberry Pi 4 HQ IMX 477 Camera. Now, I am trying to integrate the camera with a custom baseboard. I have made the corresponding device tree changes to get the camera to show up as /dev/video0.

I am using a custom Yocto image based off of OE4T’s L4T 35.1 branch.

root@jetson-xavier-nx-devkit-emmc:~# v4l2-ctl --list-devices
NVIDIA Tegra Video Input Device (platform:tegra-camrtc-ca):
	/dev/media0

vi-output, imx477 31-001a (platform:tegra-capture-vi:0):
	/dev/video0

It appears that the NVIDIA/RidgeRun IMX477 driver is properly bound as well:

root@jetson-xavier-nx-devkit-emmc:~# dmesg | grep imx
[    9.556043] imx477 31-001a: tegracam sensor driver:imx477_v2.0.6
[    9.869346] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx477 31-001a bound

However, I can not read data off the camera:

root@jetson-xavier-nx-devkit-emmc:~# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1

hangs, and the resulting frame.raw has size 0 bytes. With v4l2 debugging enabled, dmesg contains the following error:

[ 3044.365733] ------------[ cut here ]------------
[ 3044.366174] WARNING: CPU: 3 PID: 932 at drivers/media/common/videobuf2/videobuf2-core.c:1956 __vb2_queue_cancel+0x1b8/0x220
[ 3044.366634] Modules linked in: 8021q garp mrp bnep realtek iwlmvm nvgpu mac80211 snd_soc_tegra210_ope snd_soc_tegra186_asrc snd_soc_tegra210_admaif snd_soc_tegra186_arad snd_soc_tegra186_dspk snd_soc_tegra210_iqc snd_soc_tegra_pcm snd_soc_tegra210_mvc snd_soc_tegra210_dmic snd_soc_tegra210_afc snd_soc_tegra210_adx snd_soc_tegra210_amx snd_soc_tegra210_i2s snd_soc_tegra210_mixer snd_soc_tegra210_sfc aes_ce_blk crypto_simd cryptd iwlwifi aes_ce_cipher ghash_ce sha2_ce snd_soc_tegra210_adsp sha256_arm64 tegra_udrm sha1_ce snd_soc_spdif_tx snd_soc_tegra_machine_driver pwm_fan snd_soc_tegra_utils snd_soc_simple_card_utils mttcan snd_hda_codec_hdmi max77620_thermal cfg80211 snd_soc_tegra210_ahub can_dev snd_hda_tegra nvadsp btusb btrtl tegra_bpmp_thermal cdc_acm tegra210_adma btbcm snd_hda_codec btintel userspace_alert nv_imx477 snd_hda_core spi_tegra114 nvmap ina3221 fuse
[ 3044.369329] CPU: 3 PID: 932 Comm: v4l2-ctl Tainted: G        W         5.10.104-l4t-r35.1+gc94c68944656 #1
[ 3044.376837] Hardware name: Unknown NVIDIA Jetson Xavier NX Developer Kit/NVIDIA Jetson Xavier NX Developer Kit, BIOS 1.0-d7fb19b 08/10/2022
[ 3044.389422] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--)
[ 3044.395242] pc : __vb2_queue_cancel+0x1b8/0x220
[ 3044.399920] lr : __vb2_queue_cancel+0x40/0x220
[ 3044.404632] sp : ffff800019663ac0
[ 3044.407793] x29: ffff800019663ac0 x28: ffff4ddfda9c1d00
[ 3044.413561] x27: ffffc5083cf17000 x26: fffffffffffffe00
[ 3044.418814] x25: ffff4ddfd7778f68 x24: ffff4ddfda9c2300
[ 3044.424586] x23: ffff4ddfc0ee91f8 x22: ffff4ddfc0ee9080
[ 3044.429837] x21: ffff4ddfc0ee9678 x20: ffff4ddfc0ee9720
[ 3044.435523] x19: ffff4ddfc0ee9678 x18: ffffffffffffffff
[ 3044.440779] x17: 0000000000000000 x16: ffffc5083c738970
[ 3044.446555] x15: ffffffffffffffff x14: ffffff0000000000
[ 3044.451799] x13: ffffffffffffffff x12: 0000000000000003
[ 3044.457574] x11: 0101010101010101 x10: ffff7f7f7f7f7f7f
[ 3044.462831] x9 : 7f7f7f7f7f7f7f7f x8 : ffff800019663930
[ 3044.468421] x7 : 0000000000000000 x6 : 000000164e6300f5
[ 3044.473932] x5 : 00ffffffffffffff x4 : 000a0c11c52d18e9
[ 3044.479105] x3 : 0000000000000017 x2 : ffffc5083b75d090
[ 3044.484696] x1 : ffff4ddfc0ee23f0 x0 : 0000000000000004
[ 3044.489775] Call trace:
[ 3044.492235]  __vb2_queue_cancel+0x1b8/0x220
[ 3044.496272]  vb2_core_queue_release+0x30/0x60
[ 3044.500974]  _vb2_fop_release+0x88/0xa4
[ 3044.504520]  tegra_channel_close+0x68/0x180
[ 3044.508524]  v4l2_release+0xc0/0xf4
[ 3044.512023]  __fput+0x80/0x234
[ 3044.515155]  ____fput+0x24/0x30
[ 3044.518148]  task_work_run+0x88/0x140
[ 3044.521819]  do_exit+0x354/0xa34
[ 3044.525300]  do_group_exit+0x40/0xa0
[ 3044.528809]  get_signal+0x108/0x83c
[ 3044.532489]  do_notify_resume+0x188/0x990
[ 3044.536502]  work_pending+0xc/0x384
[ 3044.539740] ---[ end trace c05deafa2bc57d81 ]---
[ 3044.544637] videobuf2_common: driver bug: stop_streaming operation is leaving buf 000000007edf71cd in active state
[ 3044.554963] videobuf2_common: driver bug: stop_streaming operation is leaving buf 0000000085251835 in active state
[ 3044.565179] videobuf2_common: driver bug: stop_streaming operation is leaving buf 0000000033b02c9c in active state
[ 3044.575430] videobuf2_common: driver bug: stop_streaming operation is leaving buf 0000000062b0d257 in active state

v4l-compliance -s has the following output:

root@jetson-xavier-nx-devkit-emmc:~# v4l2-compliance -s
v4l2-compliance 1.22.1, 64 bits, 64-bit time_t

Compliance test for tegra-video device /dev/video0:

Driver Info:
	Driver name      : tegra-video
	Card type        : vi-output, imx477 31-001a
	Bus info         : platform:tegra-capture-vi:0
	Driver version   : 5.10.104
	Capabilities     : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : tegra-camrtc-ca
	Model            : NVIDIA Tegra Video Input Device
	Serial           :
	Bus info         :
	Media version    : 5.10.104
	Hardware revision: 0x00000003 (3)
	Driver version   : 5.10.104
Interface Info:
	ID               : 0x0300000b
	Type             : V4L Video
Entity Info:
	ID               : 0x00000009 (9)
	Name             : vi-output, imx477 31-001a
	Function         : V4L2 I/O
	Pad 0x0100000a   : 0: Sink
	  Link 0x0200000f: from remote pad 0x1000003 of entity '13e10000.host1x:nvcsi@15a00000-' (Unknown sub-device (0002000a)): Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-compliance.cpp(783): doioctl(node, ioc, nullptr) != ENOTTY
	test invalid ioctls: FAIL

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 13
	Standard Compound Controls: 0 Private Compound Controls: 7

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-formats.cpp(1344): ret && node->has_frmintervals
	test VIDIOC_G/S_PARM: FAIL
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(751): q.create_bufs(node, 1, &fmt) != EINVAL
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(829): ret >= 0 || err != EINVAL
	test read/write: FAIL
	test blocking wait: OK
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1398): q.create_bufs(node, 1, &fmt) != EINVAL
	test MMAP (no poll): FAIL
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1398): q.create_bufs(node, 1, &fmt) != EINVAL
	test MMAP (select): FAIL
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1398): q.create_bufs(node, 1, &fmt) != EINVAL
	test MMAP (epoll): FAIL
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1601): ret && ret != ENOTTY (got 22)
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1732): setupUserPtr(node, q)
	test USERPTR (no poll): FAIL
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1601): ret && ret != ENOTTY (got 22)
		fail: ../../../v4l-utils-1.22.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(1732): setupUserPtr(node, q)
	test USERPTR (select): FAIL
	test DMABUF: Cannot test, specify --expbuf-device

Total for tegra-video device /dev/video0: 53, Succeeded: 44, Failed: 9, Warnings: 0

So basically the camera does appear properly but any attempt to create buffers from its data fails. Does anyone have advice for further debugging this? I have not changed anything related to the camera modes in the device tree, they are the same as what worked on the devkit.

Note: I have stopped nvargus-daemon entirely to focus on debugging v4l2. The brief testing I did with argus showed a similar problem where it was unable to read data off the camera and as a result was getting stuck.

hello pwolfe1,

you may update device tree settings accordingly. could you please also review the schematic to review connection settings?

Our problem ended up being pipeline-related. Basically our schematic had a mislabeled connection as a result in our device tree our serial lanes for the cameras were swapped. As a result no data was actually getting to the VI we had told the system to expect data from our first camera interface at - leading to the timeout error message. From what I can tell it was essentially the same problem as described in this thread:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.