Nano Dev Kit B01 + Raspi HQ cam doesn't work with v4l2-ctl (JetPack 4.6)

Hello,

we’ve recently flashed a brand new Nano Dev Kit (B01) with JetPack 4.6 and configured its CSI ports to support IMX477 via the jetson-io.py. The Raspberry Pi HQ Camera (with the resistor R8 removed) can be found by v4l2-ctl and can also stream using nvarguscamerasrc.

BUT it failed streaming using v4l2-ctl. For streaming we used
v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw --verbose
and
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw --verbose.
Only in few cases, it worked. In most cases, it just hanged. While hanging, there were constantly
video4linux video0: frame start syncpt timeout!0
errors logged in the Kernel log, which means that the frame start signal on the MIPI lane was not received by the driver (if I am not mistaken).

We’ve verified that both the camera and the MIPI cable are fine. Could you please suggest us what could be the cause to this problem and how to fix it? Any help will be appreciated. Thanks!

Sorry for the late resposne, we will have the investigation to do the updat soon.

Thanks for the response. I am working with zhuda on this. Here’s some additional information for you:
We have actually tried 3 Nano Developer kits, all with JetPack 4.6 installed and on one of them the camera works well with v4l2-ctl, but on the other two we get the errors mentioned by zhuda most of the time. We have several cables and RaspI HQ cameras and have tried many combinations with our boards. We have also cloned the SD card to rule out any differences in the installation status.
It appears as if JetPack 4.6 solves the problem for other users (see Nano 2GB + Raspi HQ Cam nvarguscamerasrc working but not v4l2-ctl - #19 by JerryChang) but, at least for us, it appears to be dependent on the particular DevKit hardware being used.

What’s about the 3840x2160 this mode?

@ShaneCCC, I see the same results on two of my identical 2GB Nano dev kits with the Arducam IMX477 Mini. With the stock Nvidia drivers, both on the latest L4T, I am only able to get v4l2-ctl working with one of them. I’ve tried both sensor modes (3840x2160 and 1920x1080 that the stock Nvidia driver provides). Let me know if there’s any particular debug I can get you. I also see those same video4linux video0: frame start syncpt timeout!0 messages on the non-working rig. I had heard that v4l2 doesn’t like to work after nvarguscamerasrc has been initialized, so I tried a fresh reboot with the nvargus-daemon disabled, and got the same results. Happy to help, just let me know what I can do. Thanks!

@ShaneCCC, both 3840x2160 and 1920x1080 have the same problem as described above.

As a side note, it would be great to have the full resolution 4032x3040 30fps sensor mode available as it is in the original RidgeRun and current Arducam drivers. This 4:3 aspect ratio is extremely valuable for certain use-cases. Understandably at that throughput, v4l2 may have bandwidth issues transporting the raw bayer data. However, that sensor mode doesn’t have any issues when used in Gstreamer. Maybe there’s something different about getting raw bayer->system memory vs. raw bayer->ISP Debayer->system memory (which is what Gstreamer is doing). Even if that were the reason why v4l2 has trouble with this sensor resolution, it wouldn’t be an issue if it were possible to control the framerate via v4l2-ctl so that we could lower the bandwidth pressure, but that control doesn’t seem to take effect across any sensor mode (tested on both drivers - Nvidia and Arducam). However, unlike v4l2, Gstreamer is able to control the framerate correctly (down to 2fps and up to 60fps for 1920x1080).

I just try the dual IMX477 connect to Nano and v4l2-ctl have problem on video1 but without problem on video0.

Just wanted to check in on this. I’m still unable to use v4l2-ctl on one of my Jetson Nano’s.

@zhuda, are you still having this same issue?

@ShaneCCC: thanks for the test and response. So do you know that root cause it might be and how to fix it? Because for us both ports (video0 and video1) don’t work with streaming. The IO was configured as IMX477 dual.

@dustinkerstein: thanks for the cross check. The issue is still valid. We have the same problem on both ports.

Sure thing. You may want to untick the “Solution” button as the thread might get locked automatically.

And also, just to clarify, on the Nano 2GB Dev Kit, there is only one CSI port - which does’t work with v4l2-ctl on one of my Nanos - but the other Nano 2GB Dev Kit works fine.

1 Like

Hi Shane,

Is there anything else we can debug on these boards that aren’t working with v4l2? Is it worth building a kernel with the debug flags enabled? Thanks again.

Only v4l2-ctl doesn’t work? Does gstreamer/argus working without problem?

Yes, Shane. Only v4l2-ctl doesn’t work. gstreamer with argus works fine.

Thanks for confirm will check it next week.
May I know why do you need make v4l2-ctl working.

Shane, I can confirm only v4l2-ctl doesn’t work (Argus/Gstreamer works fine) on a fresh vanilla install of L4T 32.6.1 on one of my 2GB Nano Dev Kits (the other seemingly identical 2GB Nano Dev Kit works fine). On the broken one, I followed these steps:

  1. Format SD Card (eliminating all the partitions)
  2. Flash L4T 32.6.1
  3. sudo apt update
  4. sudo apt upgrade
  5. sudo /opt/nvidia/jetson-io/jetson-io.py Configure Jetson Nano CSI Connection → Configure for compatible hardware → Camera IMX477 → Save pin changes → Save and exit without rebooting
  6. sudo reboot
  7. sudo jetson_clocks
  8. v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 3840x2160
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
  1. v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
  1. dmesg
...
[  143.530651] video4linux video0: frame start syncpt timeout!0
[  143.738654] video4linux video0: frame start syncpt timeout!0
[  143.946664] video4linux video0: frame start syncpt timeout!0
[  144.154680] video4linux video0: frame start syncpt timeout!0
...
  1. gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 num-buffers=60 ! 'video/x-raw(memory:NVMM),width=3840,height=2160,framerate=30/1' ! nvvidconv ! nvv4l2h265enc ! h265parse ! mp4mux ! filesink location=out.mp4
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL ...
PATH :/home/nano
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 8
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 3840 H = 2160
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
NVMEDIA: H265 : Profile : 1
Got EOS from element "pipeline0".
Execution ended after 0:00:02.768813589
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Setting pipeline to NULL ...
Freeing pipeline ...
  1. dmesg
[  613.217906] misc tegra_camera_ctrl: ISO BW req 1655985 > 1500000 (max) capping to max
[  613.217913] misc tegra_camera_ctrl: tegra_camera_update_isobw: Warning, Requested ISO BW 1500000 has been capped to VI's max BW 1500000
[  615.389887] misc tegra_camera_ctrl: tegra_camera_update_isobw: Warning, Requested ISO BW 1500000 has been capped to VI's max BW 1500000
  1. Gstreamer output file out.mp4 verified to contain correct video data.

So there seems to be some difference between seemingly identical 2GB Nano Dev Kits. What kind of debug / info can I get to help isolate / fix this issue? My personal / work need for v4l2 is to enable raw bayer capture + long exposures (ie. 50+ seconds) for still photography. Using Argus isn’t a possibility for my use-case because it doesn’t offer those features currently.

1 Like

Quick follow-up, I have the same v4l2 issues when trying lower resolutions:

v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=1 --stream-mmap --stream-count=100

VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG10'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok

Results in the same dmesg output:

[ 1324.805913] video4linux video0: frame start syncpt timeout!0
[ 1325.013895] video4linux video0: frame start syncpt timeout!0
[ 1325.221919] video4linux video0: frame start syncpt timeout!0

Quick update - I was able to swap a different IMX477 module (and cable) to the broken Nano, and now I am able to get v4l2-ctl working on this Nano. So for at least my issue (not sure if it’s related to @zhuda’s issue) I have narrowed it down either to a particular IMX477 sensor module or CSI cable. Still it’s rather surprising that Gstreamer has no issues regardless. Does Argus/Gstreamer somehow use different MIPI CSI bandwidth parameters?

Have a try this patch.

diff --git a/drivers/media/i2c/imx477_mode_tbls.h b/drivers/media/i2c/imx477_mode_tbls.h
index 9e66a0e..c3ba2fa 100644
--- a/drivers/media/i2c/imx477_mode_tbls.h
+++ b/drivers/media/i2c/imx477_mode_tbls.h
@@ -29,6 +29,7 @@

 static const imx477_reg imx477_start[] = {
        {IMX477_STANDBY_REG, 0x1},
+       {IMX477_TABLE_WAIT_MS, IMX477_WAIT_MS*3},
        {IMX477_TABLE_END, 0x00}
 };

@@ -38,7 +39,9 @@ static const imx477_reg imx477_stop[] = {
 };

 static const imx477_reg imx477_mode_common[] = {
-       {IMX477_TABLE_WAIT_MS, IMX477_WAIT_MS},
+       {IMX477_TABLE_WAIT_MS, IMX477_WAIT_MS*10},
+        /* software reset */
+        {0x0103, 0x01},
        {0x0136, 0x18},
        {0x0137, 0x00},
        {0x0808, 0x02},

@ShaneCCC: Thank you so much! Your patch has made the streaming via v4l2-ctl work on both csi ports. So the solution/workaround was to increase the wait time when reading/writing to sensor registers?

@dustinkerstein: The solution provided by Shane has worked for me. How about you?