Argus Error: sensor modes do not match

I’m hoping to track down the origin of an Argus error. Here is the full report from argus_daemon.service with the error bolded.

=== Listening for connections... ===
=== Connection 7F9D6A31E0 established ===
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_center_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_center_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_center_liimx185.isp] 8/16---- imager: No override file found. ----
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_front_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_front_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_front_liimx185.isp] 8/16---- imager: No override file found. ----
CameraProvider result: provider=0x7f98b680e0, shim=0x7f98b6cf90, status=0, rpc status=1, size=9
<b>(Argus) Error InvalidState: Device provided at index 1 sensor modes do not match device at index 0 (in src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 226)</b>
/== CLEANUP 0x7f98b6cf90 ==\
  Destroying real provider 0x7f98b680e0
\== CLEANUP DONE ==/
(Argus) Error EndOfFile:  (propagating from libs/rpc_socket_server/ServerSocketManager.cpp, function recvThreadCore(), line 138)
(Argus) Error EndOfFile:  (propagating from libs/rpc_socket_server/ServerSocketManager.cpp, function run(), line 56)
=== Connection 7F9D6A31E0 closed ===
Cleaning up 0 requests...
Cleaning up 0 queues...
Cleaning up 0 streams...
Cleaning up 0 stream settings...
=== Connection 7F9D6A31E0 cleaned up ===

This results when attempting to create a UniqueObj from a vector containing two identical imx185 camera devices, as demonstrated in the Argus syncSensor sample. The Argus::status returned is 3 (STATUS_UNAVAILABLE).

The error seems to be saying that the devices are not the same and somehow have different modes. However looking at dmesg clearly shows the same driver being invoked for both devices:

[   59.770409] imx185 30-001a: imx185_power_on: power on
[   59.777904] imx185 30-001a: imx185_power_off: power off
[   59.783900] imx185 31-001a: imx185_power_on: power on
[   59.789212] imx185 31-001a: imx185_power_off: power off

And if I create an ISensorMode interface and print out both cameras modes they are identical.

I even went so far as to delete all but one mode to see if there was some strange indexing like in tegra-camera-platform where the modules are indexed backwards. But that doesn’t seem to have helped.

I’m assume there’s some sort of device tree mismatch causing this. How does Argus query the sensor modes?

Thanks

I should mention streaming each camera individually works, and using Argus to create a capture session with one device (both a vector<CameraDevice*> of size 1, and otherwise) works.

This error only occurs when I pass in a vector of size 2 with both my devices.

Could you have a try argus_camera sample APP to launch multiple camera.

I tested the following samples, all are working:

  • tegra_multimedia_api/argus/build/samples/multiSensor
  • tegra_multimedia_api/argus/build/samples/multiStream
  • tegra_multimedia_api/samples/13_multi_camera

I’ve had no issue starting the cameras as independent streams. The failure is only related to starting both together like in the syncSensor sample.

Here are the results

$ ./multi_camera -n 2
[INFO] (NvEglRenderer.cpp:109) <renderer0> Setting Screen width 640 height 480
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_center_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_center_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_center_liimx185.isp] 8/16---- imager: No override file found. ----
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_front_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_front_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_front_liimx185.isp] 8/16---- imager: No override file found. ----
Argus Version: 0.96.2 (single-process)
CONSUMER: Waiting until producer is connected...
CONSUMER: Producer has connected; continuing.
CONSUMER: Waiting until producer is connected...
CONSUMER: Producer has connected; continuing.
SCF: Error InvalidState:  NonFatal ISO BW requested not set. Requested = 2147483647 Set = 4500000 (in src/services/power/PowerServiceCore.cpp, function setCameraBw(), line 653)
SCF: Error InvalidState:  NonFatal ISO BW requested not set. Requested = 4294967294 Set = 4500000 (in src/services/power/PowerServiceCore.cpp, function setCameraBw(), line 653)
SCF: Error InvalidState:  NonFatal ISO BW requested not set. Requested = 2147628666 Set = 4500000 (in src/services/power/PowerServiceCore.cpp, function setCameraBw(), line 653)
CONSUMER: Render frame 1
DstComp rect's width must be multiple of 2, aligning the width
CONSUMER: Render frame 2
...
CONSUMER: Render frame 100
CONSUMER: Done.
$ ./argus_multisensor 
Executing Argus Sample: argus_multisensor
Argus Version: 0.96.2 (multi-process)
PRODUCER: Creating the preview stream.
PRODUCER: Launching preview consumer thread
PREVIEW CONSUMER: Creating OpenGL context.
PREVIEW CONSUMER: Connecting to EGLStream(s).
PREVIEW CONSUMER: Connected to stream(s).
PREVIEW CONSUMER: Waiting until producer(s) connect...
PRODUCER: Creating the storage stream.
JPEG CONSUMER: Waiting until producer is connected...
PRODUCER: Starting repeat capture requests.
JPEG CONSUMER: Producer has connected; continuing.
PREVIEW CONSUMER: Producer(s) connected; continuing.
PREVIEW CONSUMER: Acquired frame 1. Rendering.
PREVIEW CONSUMER: Acquired frame 2. Rendering.
PREVIEW CONSUMER: Acquired frame 3. Rendering.
JPEG CONSUMER: Captured a still image to 'Argus_0000.jpg'
...
PREVIEW CONSUMER: Acquired frame 129. Rendering.
JPEG CONSUMER: Captured a still image to 'Argus_0126.jpg'
PREVIEW CONSUMER: No more frames. Cleaning up.
JPEG CONSUMER: Captured a still image to 'Argus_0127.jpg'
PREVIEW CONSUMER: Done.
JPEG CONSUMER: No more frames. Cleaning up.
JPEG CONSUMER: Done.
PRODUCER: Done -- exiting.

Here is the result from the syncSensor example

$ ./argus_syncsensor 
Executing Argus Sample: argus_syncsensor
Argus Version: 0.96.2 (multi-process)
Error generated. /home/ww/tegra_multimedia_api/argus/samples/syncSensor/main.cpp, execute:338 Failed to get capture session interface
Segmentation fault (core dumped)

Here is the printout from argus_daemon when running the syncSensor sample (error bolded)

=== Listening for connections... ===
=== Connection 7F8315E1E0 established ===
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_center_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_center_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_center_liimx185.isp] 8/16---- imager: No override file found. ----
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_front_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_front_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_front_liimx185.isp] 8/16---- imager: No override file found. ----
CameraProvider result: provider=0x7f7cb68080, shim=0x7f7cb6cf30, status=0, rpc status=1, size=9
<b>(Argus) Error InvalidState: Device provided at index 1 sensor modes do not match device at index 0 (in src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 226)</b>
/== CLEANUP 0x7f7cb6cf30 ==\
  Destroying real provider 0x7f7cb68080
\== CLEANUP DONE ==/
(Argus) Error EndOfFile:  (propagating from libs/rpc_socket_server/ServerSocketManager.cpp, function recvThreadCore(), line 138)
(Argus) Error EndOfFile:  (propagating from libs/rpc_socket_server/ServerSocketManager.cpp, function run(), line 56)
=== Connection 7F8315E1E0 closed ===
Cleaning up 0 requests...
Cleaning up 0 queues...
Cleaning up 0 streams...
Cleaning up 0 stream settings...
=== Connection 7F8315E1E0 cleaned up ===

dmesg

[  911.037497] imx185 30-001a: imx185_power_on: power on
[  911.044808] imx185 30-001a: imx185_power_off: power off
[  911.050777] imx185 31-001a: imx185_power_on: power on
[  911.056102] imx185 31-001a: imx185_power_off: power off
[  911.063056] imx185 30-001a: imx185_power_on: power on
[  911.068357] imx185 30-001a: imx185_power_off: power off
[  911.075306] imx185 31-001a: imx185_power_on: power on
[  911.080664] imx185 31-001a: imx185_power_off: power off
[  911.096777] imx185 31-001a: imx185_power_on: power on
[  911.102182] imx185 31-001a: imx185_power_off: power off
[  911.108157] imx185 30-001a: imx185_power_on: power on
[  911.113405] imx185 30-001a: imx185_power_off: power off
[  911.119456] imx185 30-001a: imx185_power_on: power on
[  911.124904] imx185 30-001a: imx185_power_off: power off
[  911.131192] imx185 30-001a: imx185_power_on: power on
[  911.136551] imx185 30-001a: imx185_g_input_status++ g_input_status=1
[  911.143029] imx185 30-001a: imx185_power_off: power off
[  911.164122] imx185 31-001a: imx185_power_on: power on
[  911.169606] imx185 31-001a: imx185_power_off: power off
[  911.175963] imx185 31-001a: imx185_power_on: power on
[  911.181545] imx185 31-001a: imx185_power_off: power off
[  911.187685] imx185 31-001a: imx185_power_on: power on
[  911.193106] imx185 31-001a: imx185_g_input_status++ g_input_status=1
[  911.199579] imx185 31-001a: imx185_power_off: power off
[  911.724803] argus_syncsenso[6010]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000046
[  911.734480] pgd = ffffffc1eab87000
[  911.738099] [00000000] *pgd=000000020cf45003, *pud=000000020cf45003, *pmd=0000000000000000

[  911.748232] CPU: 5 PID: 6010 Comm: argus_syncsenso Not tainted 4.4.38+ #25
[  911.755189] Hardware name: quill (DT)
[  911.758958] task: ffffffc06a13f080 ti: ffffffc1ec110000 task.ti: ffffffc1ec110000
[  911.766650] PC is at 0x7f7971cb74
[  911.770003] LR is at 0x7f7971cb64
[  911.773397] pc : [<0000007f7971cb74>] lr : [<0000007f7971cb64>] pstate: 20000000
[  911.780844] sp : 0000007ffa6b4060
[  911.784159] x29: 0000007ffa6b4060 x28: 0000007ffa6b4608 
[  911.789593] x27: 000000000000000b x26: 0000000000000100 
[  911.794963] x25: 0000007f793e6198 x24: 0000000000000001 
[  911.800375] x23: 0000007f78963aa0 x22: 0000007ffa6b4200 
[  911.805728] x21: 0000000000000000 x20: 0000000000000014 
[  911.811094] x19: 0000007ffa6b40c0 x18: 000000000050d200 
[  911.816478] x17: 0000007f7971caa0 x16: 0000007f78974028 
[  911.821830] x15: 0000007f7af8b000 x14: 0000000000000000 
[  911.827166] x13: ffffffffffffffff x12: 0000000000000005 
[  911.832514] x11: 0101010101010101 x10: 0000007ffa6b42e0 
[  911.837852] x9 : 0000007ffa6b42e0 x8 : 0000007ffa6b42e0 
[  911.843191] x7 : ffffff80ffffffe8 x6 : 0000007ffa6b42c0 
[  911.848537] x5 : 0000007ffa6b4200 x4 : 0000000000000000 
[  911.853890] x3 : 0000000000000000 x2 : 0000000000000000 
[  911.859233] x1 : 00000000fbad8000 x0 : 0000007f79789568 

[  911.866074] Library at 0x7f7971cb74: 0x7f79649000 /lib/aarch64-linux-gnu/libc-2.23.so
[  911.873934] Library at 0x7f7971cb64: 0x7f79649000 /lib/aarch64-linux-gnu/libc-2.23.so
[  911.881770] vdso base = 0x7f7af8a000

Have a check the sensor mode v4l2-ctl --list-formats-exts with the DT report.

/dev/video0

$ v4l2-ctl -DT --list-formats-ext --device=/dev/video0
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, imx185 30-001a
	Bus info      : platform:15700000.vi:0
	Driver version: 4.4.38
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
VIDIOC_G_TUNER: failed: Inappropriate ioctl for device
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)

/dev/video1

$ v4l2-ctl -DT --list-formats-ext --device=/dev/video1
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, imx185 31-001a
	Bus info      : platform:15700000.vi:2
	Driver version: 4.4.38
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
VIDIOC_G_TUNER: failed: Inappropriate ioctl for device
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)

Here’s the diff

3,4c3,4
< 	Card type     : vi-output, imx185 30-001a
< 	Bus info      : platform:15700000.vi:0
---
> 	Card type     : vi-output, imx185 31-001a
> 	Bus info      : platform:15700000.vi:2

What’s the DT define the sensor mode?

Using the Leopard Imaging driver for their LI-JTX1-MIPI-ADPT board located here:

I am aware this adapter board does not officially support hardware sync, but we have modified it according to the IMX185 datasheet to do so.

I’ve attached:

  • hardware/nvidia/platform/t18x/common/kernel-dts/t18x-common-modules/tegra186-camera-li-mipi-adpt-a00.dtsi
  • hardware/nvidia/platform/t18x/common/kernel-dts/t18x-common-platforms/tegra186-quill-camera-li-mipi-adpt-a00.dtsi

Right now I have the third camera disabled since we only need two, but I’ve tried it without disabling and with deleting that third camera. Same result.
tegra186-camera-li-mipi-adpt-a00.dtsi.txt (32.3 KB)
tegra186-quill-camera-li-mipi-adpt-a00.dtsi.txt (3.07 KB)

Here is one of the cameras with mode0 shown. I have previously stripped down the driver to just be this mode0, but I reverted it back to having multiply supported modes when there was no change in the issue.

i2c@3180000 {
        tca9546@70 {
            i2c@0 {
            imx185_a@1a {
                compatible = "nvidia,imx185";

                reg = <0x1a>;
                devnode = "video0";

                /* Physical dimensions of sensor */
                physical_w = "15.0";
                physical_h = "12.5";

                sensor_model ="imx185";
                /* Define any required hw resources needed by driver */
                /* ie. clocks, io pins, power sources */

                /* Defines number of frames to be dropped by driver internally after applying */
                /* sensor crop settings. Some sensors send corrupt frames after applying */
                /* crop co-ordinates */
                post_crop_frame_drop = "0";

                /* Convert Gain to unit of dB (decibel) befor passing to kernel driver */
                use_decibel_gain = "true";

                /* if true, delay gain setting by one frame to be in sync with exposure */
                delayed_gain = "true";

                /* enable CID_SENSOR_MODE_ID for sensor modes selection */
                use_sensor_mode_id = "true";

                mode0 {/*mode IMX185_MODE_1920X1080_CROP_30FPS*/

                    mclk_khz = "37125";
                    num_lanes = "4";
                    tegra_sinterface = "serial_a";
                    discontinuous_clk = "no";
                    dpcm_enable = "false";
                    cil_settletime = "0";
                    dynamic_pixel_bit_depth = "12";
                    csi_pixel_bit_depth = "12";
                    mode_type = "bayer";
                    pixel_phase = "rggb";
                    pixel_t = "bayer_rggb";

                    active_w = "1920";
                    active_h = "1080";
                    readout_orientation = "0";
                    line_length = "2200";
                    inherent_gain = "1";
                    mclk_multiplier = "2";
                    pix_clk_hz = "74250000";

                    min_gain_val = "0"; /* dB */
                    max_gain_val = "48"; /* dB */
                    min_hdr_ratio = "1";
                    max_hdr_ratio = "1";
                    min_framerate = "1.5";
                    max_framerate = "30";
                    min_exp_time = "30";
                    max_exp_time = "660000";
                    embedded_metadata_height = "1";
                };
                ...
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        liimx185_imx185_out0: endpoint {
                            csi-port = <0>;
                            bus-width = <4>;
                            remote-endpoint = <&liimx185_csi_in0>;
                            };
                        };
                    };
                };
            };

The kernel driver report RG10 but DT is 12.

dynamic_pixel_bit_depth = "12";
                    csi_pixel_bit_depth = "12";
                    mode_type = "bayer";
                    pixel_phase = "rggb";

Also try comment out the use_sensor_mode_id = “true”;

That did it!

$ ./argus_syncsensor 
Executing Argus Sample: argus_syncsensor
Argus Version: 0.96.2 (multi-process)
PRODUCER: Creating left stream.
PRODUCER: Creating right stream.
PRODUCER: Launching disparity checking consumer
Initializing CUDA
CONSUMER: Connecting CUDA consumer to left stream
CONSUMER: Connecting CUDA consumer to right stream
CONSUMER: Waiting for Argus producer to connect to left stream.
PRODUCER: Starting repeat capture requests.
CONSUMER: Waiting for Argus producer to connect to right stream.
CONSUMER: Streams connected, processing frames.
CONSUMER: KL distance of -0.011 with  0.79 ms computing histograms and  1.32 ms spent computing distance
CONSUMER: KL distance of -0.011 with  0.91 ms computing histograms and  1.27 ms spent computing distance
CONSUMER: KL distance of -0.011 with  0.88 ms computing histograms and  1.41 ms spent computing distance
...
PRODUCER: Captures complete, disconnecting producer.
CONSUMER: KL distance of  0.004 with  0.92 ms computing histograms and  1.57 ms spent computing distance
CONSUMER: No more frames. Cleaning up.
CONSUMER: Done.
PRODUCER: Done -- exiting.

Thanks for the help!