Can't take pictures on some cameras using nvarguscamerasrc

We have 4 cameras that can take RAW pictures fine using v4l2-ctl command, but when trying to take pictures using gst-launch-1.0 nvarguscamerasrc only first camera works, the others report this error:

 SCF: Error BadParameter: Invalid GUID (in src/services/power/PowerServiceRequest.cpp, function setRequest(), line 96)
Sep 08 10:23:05 dev-kit-sd nvargus-daemon[1832]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureRecord.cpp, function makePowerRequest(), line 422)
Sep 08 10:23:05 dev-kit-sd nvargus-daemon[1832]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureRecord.cpp, function setPowerRequirements(), line 749)
Sep 08 10:23:05 dev-kit-sd nvargus-daemon[1832]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureRecord.cpp, function issueCapture(), line 472)
Sep 08 10:23:05 dev-kit-sd nvargus-daemon[1832]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1291)
Sep 08 10:23:05 dev-kit-sd nvargus-daemon[1832]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1122)

We have to restart nvargus-daemon to take pictures again with camera0.

We have a customized Jetson Xavier NX carrier board with 4 imx219 cameras. Jetpack with R35.1 Linux Kernel 5.10.104-tegra
Cameras are behind an I2C Mux
Cameras power are controlled by an I/O expander

Any ideas?

This works:
v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1

This works:
gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=0 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera0.jpeg

This doesn’t work
gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=1 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera1.jpeg

Hello @WayneWWW @JerryChang @ShaneCCC ,

Can you provide input on this?

Thanks,

hello felipeapolanco,

this looks like a fatal failure… i.e. SCF: Error BadParameter: Invalid GUID

since you’re using a customize board with 4-cam setup, could you please also share your camera device tree.
would like to check those settings within tegra-camera-platform {}.

Attaching device-tree

decompiled.dts (403.2 KB)

hello felipeapolanco,

this is incorrect. left, right were invalid settings for position property.

			module2 {
				badge = "jakku_left_RBP194";
				position = "left";
			module3 {
				badge = "jakku_right_RBP194";
				position = "right";

hence, please update position property,
since it’s 4-cam setup, you may using "topleft", "topright, "centerleft", "centerright" for camera-0 to camera-3.

you may see-also developer guide, please refer to Module Properties for details.
thanks

New error shown after updating device tree, same behavior, cam0 works but the rest doesn’t

Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error InvalidState: Timeout waiting on frame start sensor guid 5, capture sequence ID = 0 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 507)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error InvalidState:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error InvalidState: Worker thread ViCsiHw frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error Timeout:  (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 593)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: SCF: Error Timeout: Worker thread ViCsiHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)
Sep 08 10:01:48 dev-kit-sd nvargus-daemon[2570]: Module_id 30 Severity 2 : (fusa) Error: Timeout  propagating from:/capture/src/fusaViHandler.cpp 776
Sep 08 10:01:52 dev-kit-sd nvargus-daemon[2570]: SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)

hello felipeapolanco,

did you update device tree correctly?
why there’s still shown timeout failure with GUID-5. don’t you now only enable camera-0 to camera-3?

I did update the device tree, then the error changed from Invalid GUID to GUID timeout after the update. Not sure why it mentions GUID 5 if I only have 4 cameras.

Error with camera1:
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 2, capture sequence ID = 0

error with camera2
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 3, capture sequence ID = 0

error with camera3
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 5, capture sequence ID = 0

It seems like camera1 is mapped to GUID 2, camera2 is GUID 3 and camera3 is GUID 5

I did more tests by changing the port-index in the VI section and now the nvargus-daemon fails on all cameras even though I reverted my device-tree changes to the one where camera0 was working, perhaps this is an issue with nvargus and not device-tree?

Error log now shows:
camera0 → GUID 2
camera1 → GUID 3
camera2 → GUID 5
camera3 → OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree

I’m curious on this fragment of the documentation:

Port index is used to specify the connection between two different modules. For the VI (video input) module, the port index represents the input stream. For the CSI module, it represents the actual CSI port to which the camera is connected.

I have used port-index 0,1,2,3 on camera port, nvcsi and vi sections of the device tree, not sure if this is the correct assignment.

decompiled.dts (403.3 KB)

that’s incorrect. the port-index within nvcsi and tegra-capture-vi indicate the CSI port and VI stream you’re actually using.
please see-also developer guide for Port Index session.
please note that there’re only 6 VI streams, CSI-E/F and CSI-G/H are using single VI port. i.e. VI-4/5.

Can you please tell what port-index should I use for cameras connected to CSI A,B,C and D? ixm219 camera with 2-lanes configuration.

nvcsi and tegra-capture-vi please.

hello felipeapolanco,

here’s a simple diagram.

CSI     VI

A       0
B       1

C       2
D       3

E       4
F

G       5
H

each CSI brick contain two CSI ports. for example, 1st CSI brick with CSI-A and CSI-B.
CSI ports it’s capable with 1-lane/2-lane configuration. you’ll need to use CSI-A/B together for 4-lane config.
so, for your use-case.

CSI A,B,C and D? ixm219 camera with 2-lanes configuration.

it’s port-index=<0>for both nvcsi and tegra-capture-vi for using 1st IMX219 with 2-lane config on CSI-A.
repeatedly, please assign all your 4-cam by above port mapping diagram.

Based on this diagram I assume port-index 0,1,2,3 matches the same for imx219 port, CSI and VI entries.

This also explains why v4l2-ctl can take raw pictures fine.

Can you help regarding the GUID error on nvargus? I think that’s the last step to take pictures, the GUID of a camera seems to change regardless of the device-tree, at some point cam0 was working while cam1-3 were not, we made some changes to device tree and cam0 GUID became GUID 2?! then we rolled back the device-tree changes and the GUID of cam0 is stuck at GUID 2.

root@dev-kit-sd:~# gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=0 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera0.jpeg
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 3280 H = 2464
   seconds to Run    = 0
   Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
Got EOS from element "pipeline0".
Execution ended after 0:00:04.007135456
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
^C

===================================

Sep 09 01:45:36 dev-kit-sd nvargus-daemon[4298]: === gst-launch-1.0[4299]: Connection closed (FFFF8C4B4900)SCF: Error InvalidState: Timeout waiting on frame start sensor guid 2, capture sequence ID = 0 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 507)
Sep 09 01:45:36 dev-kit-sd nvargus-daemon[4298]: SCF: Error InvalidState:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)

After trial and error, I found out that the GUID is assigned based on the “position” parameter of the tegra-camera-platform device, in order to get GUID 0,1,2,3 for cam 0,1,2,3 I used bottomleft, centerleft, centerright, topleft in that order.

I got camera0 back working but cam1-3 are still not working.

gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=1 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera0.jpeg

gets me:

=== gst-launch-1.0[2497]: Connection closed (FFFF7EB7B900)SCF: Error InvalidState: Timeout waiting on frame start sensor guid 1, capture sequence ID = 0

same for cam2 and cam3.

I can take RAW pictures using v4l2-ctl so I’m confident the hardware is fine.

Tracing logs:

CaptureSchedule-2875    [005] ....   581.905456: tegra_channel_open: vi-output, imx219 31-0010
 CaptureSchedule-2875    [005] ....   581.908196: tegra_channel_set_power: imx219 31-0010 : 0x1
 CaptureSchedule-2875    [005] ....   581.908211: camera_common_s_power: status : 0x1
 CaptureSchedule-2875    [005] ....   581.918681: tegra_channel_set_power: 13e10000.host1x:nvcsi@15a00000- : 0x1
 CaptureSchedule-2875    [005] ....   581.918687: csi_s_power: enable : 0x1
 CaptureSchedule-2875    [005] ....   581.918694: tegra_channel_set_stream: enable : 0x1
 CaptureSchedule-2875    [005] ....   581.919850: tegra_channel_set_stream: 13e10000.host1x:nvcsi@15a00000- : 0x1
 CaptureSchedule-2875    [005] ....   581.919854: csi_s_stream: enable : 0x1
 CaptureSchedule-2875    [005] ....   581.919856: tegra_channel_set_stream: imx219 31-0010 : 0x1
     kworker/0:2-85      [000] ....   581.972632: rtcpu_vinotify_event: tstamp:19017282854 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:608552487808 data:0xcd9c550010000000
     kworker/0:2-85      [000] ....   581.972636: rtcpu_vinotify_event: tstamp:19017283004 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:608552499616 data:0x0000000031000001
     kworker/0:2-85      [000] ....   582.028681: rtcpu_vinotify_event: tstamp:19018978442 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:608600320896 data:0xcd9c520010000000
     kworker/0:2-85      [000] ....   582.028685: rtcpu_vinotify_event: tstamp:19018978590 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:608600333568 data:0x0000000031000002
<==== THIS IS THE END =====>

Any ideas?

hello felipeapolanco,

please try configure enableCamInfiniteTimeout and test with Argus pipeline.
for example,

$ sudo pkill nvargus-daemon
$ export enableCamInfiniteTimeout=1
$ sudo nvargus-daemon &
$ gst-launch-1.0 nvarguscamerasrc .... 

BTW, is it possible for moving to latest release, i.e. JP-5.1/ l4t-r35.2.1

I will check with the team regarding updating to latest release.
Same error, here is the log:

root@dev-kit-sd:~# export enableCamInfiniteTimeout=1
root@dev-kit-sd:~# echo $enableCamInfiniteTimeout
1
root@dev-kit-sd:~# sudo nvargus-daemon &
[1] 14154
root@dev-kit-sd:~# === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...
root@dev-kit-sd:~#
root@dev-kit-sd:~#
root@dev-kit-sd:~#
root@dev-kit-sd:~#
root@dev-kit-sd:~# gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=2 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera0.jpeg
=== gst-launch-1.0[14156]: Connection established (FFFF96DDA900)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
---- imager: No override file found. ----
---- imager: No override file found. ----
---- imager: No override file found. ----
=== gst-launch-1.0[14156]: CameraProvider initialized (0xffff905cba70)Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 2
   Camera mode  = 0
   Output Stream W = 3280 H = 2464
   seconds to Run    = 0
   Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
Got EOS from element "pipeline0".
Execution ended after 0:00:04.007198232
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 2, capture sequence ID = 0 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 507)
SCF: Error InvalidState:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
SCF: Error InvalidState: Worker thread ViCsiHw frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)
SCF: Error Timeout:  (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 593)
SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
SCF: Error Timeout: Worker thread ViCsiHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)
^C=== gst-launch-1.0[14156]: Connection closed (FFFF96DDA900)
root@dev-kit-sd:~# SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc

For Comparison, this is cam0 log which is working fine:

root@dev-kit-sd:~# sudo nvargus-daemon &
[1] 14237
root@dev-kit-sd:~# === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...
root@dev-kit-sd:~#
root@dev-kit-sd:~# gst-launch-1.0 nvarguscamerasrc num-buffers=3 sensor-id=0 awblock=true aelock=true ! 'video/x-raw(memory:NVMM),width=3840,height=2180,framerate=21/1' ! nvvidconv flip-method=3 ! nvjpegenc ! multifilesink location=./camera0.jpeg
=== gst-launch-1.0[14239]: Connection established (FFFF83189900)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
---- imager: No override file found. ----
---- imager: No override file found. ----
---- imager: No override file found. ----
=== gst-launch-1.0[14239]: CameraProvider initialized (0xffff7cb74ab0)Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
SCF: Error BadValue: NvPHSSendThroughputHints (in src/common/CameraPowerHint.cpp, function sendCameraPowerHint(), line 56)
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 3280 H = 2464
   seconds to Run    = 0
   Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Got EOS from element "pipeline0".
Execution ended after 0:00:01.438871561
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Freeing pipeline ...
=== gst-launch-1.0[14239]: CameraProvider destroyed (0xffff7cb74ab0)=== gst-launch-1.0[14239]: Connection closed (FFFF83189900)=== gst-launch-1.0[14239]: Connection cleaned up (FFFF83189900)

hello felipeapolanco,

could you please try v4l standard controls for each of camera.
for example,
$ v4l2-ctl -d /dev/videoX --set-fmt-video=width=3840,height=2464,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
whereas X= 0,1,2,3 for your all four cameras.

We just found out we had to update the tegra_sinterface parameter on each camera mode to serial_a,b,c,d depending on the camera.
We got cam2,3 working after this.

We still have cam1 not working, stating same timeout even after correcting the tegra_sinterface parameter.

anything else we need to consider?

decompiled.dts (402.4 KB)

We applied our device tree into another Jetson and all 4 cameras worked, seems like cam1 issue is hardware related.

Updating the tegra_sinterface to the correct channel name did the trick.

Thanks.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.