Argus Sample syncSensor Fails

TX2, Jetpack 3.3, L4T 28.2.1

Running syncSensor sample produces the following result:

Executing Argus Sample: argus_syncsensor
Argus Version: 0.96.2 (multi-process)
Error generated. /home/nvidia/tegra_multimedia_api/argus/samples/syncSensor/main.cpp, execute:338 Failed to get capture session interface

The multi-sensor sample works as expected. I’m using two Leopard Imaging IMX185 cameras and their MIPI interface board.

I tried to get syncSensor to work several months ago with a different sensor and had the same issue, so I just assumed it was a camera driver problem. But apparently not. Has anyone gotten syncSensor to work?

There are no messages printed on dmesg or nvcamera-daemon

Hi Atrer,

If the adapter board you are using is LI-JTX1-MIPI-APT, this camera kit won’t support camera sync function.
We have another camera kit LI-TX1-CB-6CAM + IMX185 kit which supports camera sync function. The LI-TX1-CB-6CAM board supports up to 6 cameras and includes a FPGA which can generate trigger signals to sync the cameras. We have TX2 driver for this kit.

Ok, we have hacked the LI-JTX1-MIPI-APT to connect the camera sync signals as discussed in the IMX185 datasheet. We also cut a trace that seemed to be forcing the slave/master select pin to master mode, so now we can use the dip switch on the IMX185 to switch it between slave and master.

That seems to be working on the surface. You can stream both cameras without issue if they are both set to master. And then if you set the other one to slave it will only stream if you start the master camera first.

However I still get the same error. Passing in “Status” to Argus::ICameraProvider::createCaptureSession shows that the error in question is ‘STATUS_UNAVAILABLE’ which is mentioned in the docs:

STATUS_UNAVAILABLE will be placed into status if any of the devices are already in use.

I am running with sudo, and it works when using only one device (both a vector<CameraDevice*> of size 1, and otherwise). So I’m not sure what’s up.

Hi Atrer,

Good to know you can stream two videos via master/slave mode. Does the video have any issues when you see the error log?

The video never starts. After the error I fail out gracefully since we are unable to create the Argus ICaptureSession interface (to configure/start the output streams), I don’t see anything else coming from Argus.

There are no messages on the nvcamera-daemon, Argus never tries to connect to it. I’m running the daemon with logging like below:

sudo service nvcamera-daemon stop
enableCamScfLogs=1
enableCamPclLogs=1
sudo /usr/sbin/nvcamera-daemon

Nothing shows on dmesg either.

If you’re asking about streaming master/slave mode separately. If I start the slave-mode camera without the master running (using a gstreamer pipeline) no frames come out and nvcamera-daemon complains about the captures timing out. But if I then start the master camera streaming (using a separate gstreamer pipeline) the slave begins producing images and the errors abate.

Hi Atrer,

We have a few questions.

  1. Before the hardware rework, can the two master cameras stream out videos?
  2. For the hardware rework, did you just connect the XHS and XVS of two IMX185 boards (through adapter board)?
  3. After the hardware rework, can you still get video output with two master cameras?
  1. Yes, we have several of the LI-JTX1-MIPI-APT boards with IMX185 cameras and have had no issues streaming each camera. The board in question was tested and working before we modified it.

  2. Yes, we connected XHS and XVS. We also removed a pulldown resistor from the LI-JTX1-MIPI-APT that was forcing both cameras to master mode (in spite of the DIP switch on the IMX185 board itself).

  3. Yes, with the master/slave DIP switch set to “ON” the two cameras behave as normal. We can start/stop both streams independently or stream them both together on separate pipelines. When we clear the DIP switch on our chosen ‘slave’ camera it behaves as expected. Trying to stream the slave camera without the master results in frame timeouts on nvcamera-daemon. If we start the master first, and then stream the slave it works fine.

Hi Atrer,

Thanks for the info.
With master/slave mode, can you capture raw data via v4l2? You can try below command.

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --set-ctrl sensor_mode=1 --stream-mmap --stream-count=3 --stream-to=IMX185_10bit.raw -d /dev/video0

Which driver are using? I assume you didn’t modify the driver.
For " If we start the master first, and then stream the slave it works fine." in your comments #9, which software are you using to check the stream even there is no video preview? I assume neither master nor slave camera have video preview.

We are using the tri_streaming_imx185 driver with the third camera disabled.

diff --git a/kernel-dts/t18x-common-modules/tegra186-camera-li-mipi-adpt-a00.dtsi b/kernel-dts/t18x-common-modules/tegra186-camera-li-mipi-adpt-a00.dtsi
index 88dd50c..3814953 100644
--- a/kernel-dts/t18x-common-modules/tegra186-camera-li-mipi-adpt-a00.dtsi
+++ b/kernel-dts/t18x-common-modules/tegra186-camera-li-mipi-adpt-a00.dtsi
@@ -41,6 +41,7 @@
                                port@2 {
                                        reg = <2>;
                                        liimx185_vi_in2: endpoint {
+                                               status = "disabled";
                                                csi-port = <0>;
                                                bus-width = <4>;
                                                remote-endpoint = <&liimx185_csi_out2>;
@@ -96,6 +97,7 @@
                                };
                        };
                        channel@2 {
+                               status = "disabled";
                                reg = <2>;
                                ports {
                                        #address-cells = <1>;
@@ -780,6 +782,7 @@
                        };
                        i2c@2 {
                        imx185_e@1a {
+                               status = "disabled";
                                compatible = "nvidia,imx185";
 
                                reg = <0x1a>;
@@ -1185,6 +1188,7 @@
                                };
                        };
                        module0 {
+                               status = "disabled";
                                badge = "imx185_bottom_liimx185";
                                position = "bottom";
                                orientation = "1";

I looked over the 6_streaming_imx185 driver and didn’t see anything special in the device tree. There were a few differences in the kernel patch (especially the mode tables). But according to dmesg imx185_s_stream isn’t being called, making me think it’s a device tree setting somewhere.

Here is the dmesg log with DEBUG enabled in the driver when attempting the Argus sync. Notice the stream is never started, meaning the mode tables have not been sent yet.

[  159.830426] imx185 30-001a: imx185_power_on: power on
[  159.837541] imx185 30-001a: imx185_power_off: power off
[  159.843823] imx185 31-001a: imx185_power_on: power on
[  159.849109] imx185 31-001a: imx185_power_off: power off
[  159.855928] imx185 30-001a: imx185_power_on: power on
[  159.861417] imx185 30-001a: imx185_power_off: power off
[  159.867564] imx185 31-001a: imx185_power_on: power on
[  159.873049] imx185 31-001a: imx185_power_off: power off
[  159.899754] imx185 31-001a: imx185_power_on: power on
[  159.905007] imx185 31-001a: imx185_power_off: power off
[  159.911023] imx185 30-001a: imx185_power_on: power on
[  159.916290] imx185 30-001a: imx185_power_off: power off
[  159.922314] imx185 30-001a: imx185_power_on: power on
[  159.927770] imx185 30-001a: imx185_power_off: power off
[  159.933709] imx185 30-001a: imx185_power_on: power on
[  159.939033] imx185 30-001a: imx185_power_off: power off
[  159.953750] imx185 31-001a: imx185_power_on: power on
[  159.959003] imx185 31-001a: imx185_power_off: power off
[  159.965177] imx185 31-001a: imx185_power_on: power on
[  159.970532] imx185 31-001a: imx185_power_off: power off
[  159.976537] imx185 31-001a: imx185_power_on: power on
[  159.981847] imx185 31-001a: imx185_power_off: power off

Here’s the dmesg log from that v4l2-ctl command above using video0 (set to master). IMX185_10bit.raw has 12441600 bytes and xxd shows it isn’t all zeroes, so I believe that was successful.

[  464.952670] imx185 30-001a: imx185_power_on: power on
[  464.983006] imx185 30-001a: imx185_s_stream++ enable 1
[  465.506335] imx185 30-001a: imx185_s_stream++ enable 0
[  465.513896] tegra-i2c 3180000.i2c: no acknowledge from address 0x1a
[  465.533998] imx185 30-001a: imx185_power_off: power off

Running it on video1 (set to slave) still produces IMX185_10bit.raw with 12441600, but every byte is 0x00. Here’s the dmesg showing the timeouts.

[  640.590766] imx185 31-001a: imx185_power_on: power on
[  640.622891] imx185 31-001a: imx185_s_stream++ enable 1
[  641.919230] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  642.923307] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  643.927375] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  644.931419] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  645.935482] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  646.939552] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  647.943607] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[  647.949843] imx185 31-001a: imx185_s_stream++ enable 0
[  647.958127] tegra-i2c 3180000.i2c: no acknowledge from address 0x1a
[  647.991429] imx185 31-001a: imx185_power_off: power off

As for my comments in #9 I am using a gst-launch pipeline to stream the video to VLC media player. Tell VLC to “Open Network Stream” and use ‘rtp://@:PORT’

# Define INDEX, HOST, PORT above
    FPS=30
    STREAM_WIDTH=1920
    STREAM_HEIGHT=1080

    gst-launch-1.0 -v \
      nvcamerasrc auto-exposure=2 do-timestamp=true sensor_id=$INDEX fpsRange="$FPS $FPS" intent=3 \
        ! "video/x-raw(memory:NVMM), width=(int)$STREAM_WIDTH, height=(int)$STREAM_HEIGHT, format=(string)I420, framerate=(fraction)$FPS/1" \
        ! omxh264enc \
        ! "video/x-h264, stream-format=(string)byte-stream" \
        ! h264parse \
        ! mpegtsmux \
        ! rtpmp2tpay \
        ! udpsink host=$HOST port=$PORT sync=false async=false

I commented out the reset_gpio toggling in imx185_power_on and imx185_power_off as shown in the 6_streaming kernel patch, but that had no effect.

Think I found something

https://devtalk.nvidia.com/default/topic/1039183/jetson-tx2/argus-syncing-multiple-capture-sessions/post/5279994/#5279994

I’m going to take a closer look at the 6_sensor sync’d driver and see if I can figure out how to rig up the second camera as a slave of the first.

Under t18x-common-platforms/tegra186-quill-camera-imx185-a00.dtsi the 6_stream driver adds ‘force_bus_start = <CAMERA_I2C_MUX_BUS(10)>;’ to the tca9548@77 controlling its cameras.

The tri_stream driver has ‘force_bus_start = <CAMERA_I2C_MUX_BUS(0)>’ but it was not explicitly added by the driver. It was already added to the t18x-common-platforms/tegra186-quill-camera-li-mipi-adpt-a00.dtsi file by Nvidia.

There are 3 other references to force_bus_start in the e3333, e3322, and imx274 drivers respectively, but they are all ‘force_bus_start = <CAMERA_I2C_MUX_BUS(0)>’

I’ll look through the sensor driver programming guide and see if I can find out what that is.

Edit: No luck, not referenced in the docs.

Looks like force_bus_start is used in drivers/i2c/muxes/i2c-mux-pca954x.c, but it seems to just be the starting address for creating i2c adapters.

The macro is just adding 30 (0x1e) to the number which seems to be related to the module devname. Normally they start 30, 31, 32, etc. But the 6_stream driver has them incrementing from 40. Hence the 10 in that macro.

Hi Atrer,

We are not sure whether the hardware reworked correctly or not.
One way to confirm this is to add two jumpers between two IMX185 cameras directly(T1 --> T1 (XVS), T3 --> T3 (XHS)). Then use an un-reworked LI-JTX1-MIPI-ADPT and our TX2 driver to test them. You don’t have to disable the third camera.

My issue has to do with Argus opening both cameras together like in the syncSensor sample. At that point the cameras have not been configured and streaming has not started. The hardware being set up correctly or not should only be a problem later once I’m trying to start the stream. That’s why I’m thinking it is something in the device tree.

The mode tables have not been sent and the cameras should still be in reset at this point.

Using debug instructions from here:
https://devtalk.nvidia.com/default/topic/1027839/jetson-tx2/errors-in-the-syscsensor-example-in-the-argus-folder/post/5228051/#5228051

=== Listening for connections... ===
=== Connection 7F7B7D81E0 established ===
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_center_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_center_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_center_liimx185.isp] 8/16---- imager: No override file found. ----
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx185_front_liimx185.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx185_front_liimx185.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx185_front_liimx185.isp] 8/16---- imager: No override file found. ----
CameraProvider result: provider=0x7f74b541b0, shim=0x7f74b59110, status=0, rpc status=1, size=9
<b>(Argus) Error InvalidState: Device provided at index 1 sensor modes do not match device at index 0 (in src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 226)</b>

So it has to do with the sensor modes. I’ll try just using the modes from the 6_sensor driver.

No luck,

Made a thread on the error itself to see if @ShaneCCC can help
https://devtalk.nvidia.com/default/topic/1055762/jetson-tx2/argus-error-sensor-modes-do-not-match/

Shane solved it: https://devtalk.nvidia.com/default/topic/1055762/jetson-tx2/argus-error-sensor-modes-do-not-match/post/5353814/#5353814

The sensor driver was reporting RG10 format but the default mode was not one of the 10-bit modes. I didn’t want to take any chances and just deleted all but the 10-bit modes. Though I expect if I just reordered them and made the default a 10-bit it would probably pass the check.

I will do some tests and see if our hardware modifications are working properly, though I expect I’m going to need to modify the driver and put in some way to specify which camera is a slave so we can set the internal registers differently.