Hi all!
Background:
I’m using a Seedstudio A203V2 carrier board with my NX, since L4T 35.1.0 drivers have been recently published on the producer’s webpage.
I also bought several arducam MIPI-CSI IMX477 and IMX219 cameras to test, but I’m facing inconsistent results with my capture script that uses openCV to show the stream.
Starting from the a fresh install, I usually start extending my storage, using rootOnTF script to load the SD card as rootfs; I then load CUDA 11.4 from the SDKManager, install NoMachine to remotely connect, Torch, Tensorrt, etc.
I usually have no particular problems with opening the script with a fresh system, but I eventually finish breaking something that is preventing me to show the camera.
Details:
nvidia@nvidia-a208:~$ cat /etc/nv_tegra_release
R35 (release), REVISION: 1.0, GCID: 31346300, BOARD: t186ref, EABI: aarch64, DATE: Thu Aug 25 18:41:45 UTC 2022
Pipeline (from arducam.py):
def gstreamer_pipeline(
self,
capture_width=1280,
capture_height=720,
display_width=1280,
display_height=720,
framerate=60,
flip_method=0,
):
return (
"nvarguscamerasrc ! "
"video/x-raw(memory:NVMM), "
"width=(int)%d, height=(int)%d, "
"format=(string)NV12, framerate=(fraction)%d/1 ! "
"nvvidconv flip-method=%d ! "
"video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
"videoconvert ! "
“video/x-raw, format=(string)BGR ! appsink”
% (
capture_width,
capture_height,
framerate,
flip_method,
display_width,
display_height,
)
)
The camera is recognized by v4l2:
nvidia@nvidia-a208:~$ v4l2-ctl -d /dev/video0 --all
Driver Info:
Driver name : tegra-video
Card type : vi-output, imx219 2-0010
Bus info : platform:tegra-capture-vi:0
Driver version : 5.10.104
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : tegra-camrtc-ca
Model : NVIDIA Tegra Video Input Device
Serial :
Bus info :
Media version : 5.10.104
Hardware revision: 0x00000003 (3)
Driver version : 5.10.104
Interface Info:
ID : 0x03000008
Type : V4L Video
Entity Info:
ID : 0x00000006 (6)
Name : vi-output, imx219 2-0010
Function : V4L2 I/O
Pad 0x01000007 : 0: Sink
Link 0x0200000c: from remote pad 0x1000003 of entity ‘13e10000.host1x:nvcsi@15a00000-’: Data, Enabled
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
Width/Height : 3280/2464
Pixel Format : ‘RG10’ (10-bit Bayer RGRG/GBGB)
Field : None
Bytes per Line : 6560
Size Image : 16163840
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :Camera Controls
group_hold 0x009a2003 (bool) : default=0 value=0 flags=execute-on-write
sensor_mode 0x009a2008 (int64) : min=0 max=5 step=1 default=0 value=0 flags=slider
gain 0x009a2009 (int64) : min=16 max=170 step=1 default=16 value=16 flags=slider
exposure 0x009a200a (int64) : min=13 max=683709 step=1 default=2495 value=13 flags=slider
frame_rate 0x009a200b (int64) : min=2000000 max=21000000 step=1 default=21000000 value=2000000 flags=slider
sensor_configuration 0x009a2032 (u32) : min=0 max=4294967295 step=1 default=0 [22] flags=read-only, volatile, has-payload
sensor_mode_i2c_packet 0x009a2033 (u32) : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
sensor_control_i2c_packet 0x009a2034 (u32) : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
0: 0 (0x0)
1: 1 (0x1)
override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
0: 0 (0x0)
1: 1 (0x1)
height_align 0x009a2066 (int) : min=1 max=16 step=1 default=1 value=1
size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
0: 1 (0x1)
1: 65536 (0x10000)
2: 131072 (0x20000)
write_isp_format 0x009a2068 (int) : min=1 max=1 step=1 default=1 value=1
sensor_signal_properties 0x009a2069 (u32) : min=0 max=4294967295 step=1 default=0 [30][18] flags=read-only, has-payload
sensor_image_properties 0x009a206a (u32) : min=0 max=4294967295 step=1 default=0 [30][16] flags=read-only, has-payload
sensor_control_properties 0x009a206b (u32) : min=0 max=4294967295 step=1 default=0 [30][36] flags=read-only, has-payload
sensor_dv_timings 0x009a206c (u32) : min=0 max=4294967295 step=1 default=0 [30][16] flags=read-only, has-payload
low_latency_mode 0x009a206d (bool) : default=0 value=0
preferred_stride 0x009a206e (int) : min=0 max=65535 step=1 default=0 value=0
sensor_modes 0x009a2082 (int) : min=0 max=30 step=1 default=30 value=5 flags=read-only
At the moment, when I run my script, I get “Failed to create FrameConsumer” from Argus.
I’m using an Arducam IMX219 for testing
nvidia@nvidia-a208:~/drone-racing-px4/drone_racing/src/camera$ python arducam.py
GST_ARGUS: Creating output stream
(Argus) Error BadParameter: (propagating from src/eglstream/FrameConsumerImpl.cpp, function initialize(), line 89)
(Argus) Error BadParameter: (propagating from src/eglstream/FrameConsumerImpl.cpp, function create(), line 44)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadInitialize:320 Failed to create FrameConsumer
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:241 (propagating)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, waitRunning:203 Invalid thread state 3
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:806 (propagating)
[ WARN:0] global /home/ubuntu/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
This happens on both physical display (:0 with a LG monitor) but also with a virtual display I create with NoMachine (:1001 without any EDID nor physical HDMI).
This is a journal I took testing the script with a physical monitor:
nvidia@nvidia-a208:~$ cat journal | grep argus
Oct 28 13:00:39 nvidia-a208 nvargus-daemon[1235]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections…=== python[3277]: Connection established (FFFF97118900)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Oct 28 13:00:39 nvidia-a208 nvargus-daemon[1235]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Oct 28 13:00:39 nvidia-a208 nvargus-daemon[1235]: ---- imager: No override file found. ----
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: === python[3277]: CameraProvider initialized (0xffff9076dfd0)SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
Oct 28 13:00:40 nvidia-a208 nvargus-daemon[1235]: E/ libnvphs: Error: NvPHSSendThroughputHints[usecase=camera, hint=MinCPU, value=4294967295, timeout_ms=1000]: queue_or_send() failed
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvCamV4l2) Error IoctlFailed: (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function streamControl(), line 1661)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvOdmDevice) Error IoctlFailed: (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function apply(), line 206)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvCamV4l2) Error IoctlFailed: (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function streamControl(), line 1661)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvOdmDevice) Error IoctlFailed: (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function apply(), line 206)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvCamV4l2) Error IoctlFailed: (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function streamControl(), line 1661)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvOdmDevice) Error IoctlFailed: (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function apply(), line 206)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvCamV4l2) Error IoctlFailed: (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function streamControl(), line 1661)
Oct 28 13:00:41 nvidia-a208 nvargus-daemon[1235]: (NvOdmDevice) Error IoctlFailed: (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function apply(), line 206)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error InvalidState: Timeout waiting on frame start sensor guid 1, capture sequence ID = 0 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 507)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error InvalidState: (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error InvalidState: Worker thread ViCsiHw frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error Timeout: (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 593)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error Timeout: (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: SCF: Error Timeout: Worker thread ViCsiHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)
Oct 28 13:00:46 nvidia-a208 nvargus-daemon[1235]: Module_id 30 Severity 2 : (fusa) Error: Timeout propagating from:/capture/src/fusaViHandler.cpp 776
Oct 28 13:00:49 nvidia-a208 nvargus-daemon[1235]: SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Oct 28 13:00:49 nvidia-a208 nvargus-daemon[1235]: Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
Oct 28 13:01:45 nvidia-a208 nvargus-daemon[1235]: SCF: Error InvalidState: 2 buffers still pending during EGLStreamProducer destruction (propagating from src/services/gl/EGLStreamProducer.cpp, function freeBuffers(), line 302)
Oct 28 13:01:45 nvidia-a208 nvargus-daemon[1235]: SCF: Error InvalidState: (propagating from src/services/gl/EGLStreamProducer.cpp, function ~EGLStreamProducer(), line 50)
Oct 28 13:01:50 nvidia-a208 nvargus-daemon[1235]: waitForIdleLocked remaining request 102
Oct 28 13:01:50 nvidia-a208 nvargus-daemon[1235]: waitForIdleLocked remaining request 101
Oct 28 13:01:50 nvidia-a208 nvargus-daemon[1235]: SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 909)
Oct 28 13:01:50 nvidia-a208 nvargus-daemon[1235]: (Argus) Error Timeout: (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 169)
With NX the error is similar:
nvidia@nvidia-a208:~$ cat journalNX | grep argus
أكت 28 14:13:20 nvidia-a208 nvargus-daemon[1398]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections…=== python[5761]: Connection established (FFFFA9C2A900)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
أكت 28 14:13:20 nvidia-a208 nvargus-daemon[1398]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
أكت 28 14:13:20 nvidia-a208 nvargus-daemon[1398]: ---- imager: No override file found. ----
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: === python[5761]: CameraProvider initialized (0xffffa476e080)SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
أكت 28 14:13:21 nvidia-a208 nvargus-daemon[1398]: E/ libnvphs: Error: NvPHSSendThroughputHints[usecase=camera, hint=MinCPU, value=4294967295, timeout_ms=1000]: queue_or_send() failed
أكت 28 14:13:22 nvidia-a208 nvargus-daemon[1398]: (Argus) Error InvalidState: Unknown stream deleted. (in src/api/CaptureSessionImpl.cpp, function outputStreamDeleted(), line 854)
In both cases I see video0 initialized:
nvidia@nvidia-a208:~$ sudo dmesg | grep imx
[ 15.183036] imx219 2-0010: tegracam sensor driver:imx219_v2.0.6
[ 15.199481] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx219 2-0010 boundnvidia@nvidia-a208:~$ ls /dev/video0
/dev/video0
I also see fps with the current command:
nvidia@nvidia-a208:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<< 21.19 fps
<<<<<<<<<<<<<<<<<<<<< 21.19 fps
<<<<<<<<<<<<<<<<<<<<< 21.19 fps
<<<<<<<<<<<<<<<<<<<<< 21.19 fps
<<<<<<<<<<<<<<
I’ll attach both dmesg and journalctl of a test with the physical monitor and one with the virtual NX.
I also tried with argus oneShot:
nvidia@nvidia-a208:~/argus/build/samples/oneShot$ ./argus_oneshot
Executing Argus Sample: argus_oneshot
Argus Version: 0.98.3 (multi-process)
Capturing from device 0 using sensor mode 0 (3280x2464)
(Argus) Error BadParameter: (propagating from src/eglstream/FrameConsumerImpl.cpp, function initialize(), line 89)
(Argus) Error BadParameter: (propagating from src/eglstream/FrameConsumerImpl.cpp, function create(), line 44)
Failed to initialize Consumer
In case you need this one is a log where I could see the videostream, so the camera is not broken.
nvidia@nvidia-a208:~/drone-racing-px4/drone_racing/src/camera$ python3 arducam.py
nvbuf_utils: Could not get EGL display connection
No EGL Display
nvbufsurftransform: Could not get EGL display connection
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 4
Output Stream W = 1280 H = 720
seconds to Run = 0
Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0] global /home/ubuntu/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Gtk-Message: 19:27:35.029: Failed to load module “canberra-gtk-module”
Traceback (most recent call last):
File “arducam.py”, line 110, in
arducam.show_camera(False, 2)
File “arducam.py”, line 72, in show_camera
while cv2.getWindowProperty(“CSI Camera”, 0) >= 0:
cv2.error: OpenCV(4.5.4) /home/ubuntu/build_opencv/opencv/modules/highgui/src/window_gtk.cpp:805: error: (-27:Null pointer) NULL window in function ‘cvGetModeWindow_GTK’GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Sometimes I get a different error; I can run the script but it hangs without opening the GUI:
(Attempt with physical monitor below)
nvidia@nvidia-a208:~/drone-racing-px4/drone_racing/src/camera$ python arducam.py
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 4
Output Stream W = 1280 H = 720
seconds to Run = 0
Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
[ WARN:0] global /home/ubuntu/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Gtk-Message: 15:26:10.257: Failed to load module “canberra-gtk-module”
Traceback (most recent call last):
File “arducam.py”, line 110, in
arducam.show_camera(False, 2)
File “arducam.py”, line 74, in show_camera
cv2.imshow(“CSI Camera”, img)
cv2.error: OpenCV(4.5.4) /home/ubuntu/build_opencv/opencv/modules/core/src/array.cpp:2494: error: (-206:Bad flag (parameter or structure field)) Unrecognized or unsupported array type in function ‘cvGetMat’GST_ARGUS: Cleaning up
(Argus) Error Timeout: (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 137)
(Argus) Error Timeout: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error Timeout: (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 137)
(Argus) Error Timeout: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
GST_ARGUS: Done Success
I think I’m missing something here, but I got lost.
Logs:
journalNX (34.9 KB)
journal (29.5 KB)
dmesgPhy (69.6 KB)
dmesgNX (67.5 KB)
arducam.py (3.3 KB)
Thank you!