Hi all
Here we are using software from the nvidia-l4t-jetson-multimedia package: argus_multi_camera (modified to support camera selection and And displaying traces with uptime (like dmesg).).
Using : Jetpack 5.1.2
We are launching an argus_multi_camera application for each available imx715 sensor (in this case, 2 sensors).
Our endurance test consists of continuously performing the following process:
- Powering on the sensors
- Loading the drivers
- Launching an individual argus_multi_camera application for each available imx715 sensor for each sensor (In our case 2 devices) to capture and display 100 frames at 27 fps
- Unloading the drivers
- Powering off
This test works correctly for a while, but then randomly the applications stop with an error:
at uint32_t captureId = iCaptureSession->capture(request, 1800 * 1000000, &status);
- Capture frame 14: ARGUS::STATUS_TIMEOUT
In the logs provided by dmesg, the information for the imx715 driver is as follows:
[ 796.177974] imx715 10-001a: initializing mipi...
[ 796.178373] imx715 10-001a: tegracam sensor driver:imx715_v2.0.6
[ 796.178551] imx715 10-001a: [S***] imx715_board_setup++
[ 796.178792] imx715 10-001a: imx715_power_on: power on
[ 796.210641] imx715 10-001a: imx715_calculate_line_time:++
[ 796.211468] imx715 10-001a: imx715_calculate_line_time: hmax: 550 [inck], INCK: 24000000 [Hz], line_time: 7407 [ns]
[ 796.211472] imx715 10-001a: imx715_power_off: power off
[ 796.211683] imx715 10-001a: imx715_set_operation_mode: set operation mode 0
[ 796.211693] imx715 10-001a: imx715_set_data_rate:++
[ 796.211695] imx715 10-001a: imx715_verify_data_rate++
[ 796.211698] imx715 10-001a: imx715_verify_data_rate: Selected data rate is not supported with 2 CSI lane mode, switching to default!
[ 796.211700] imx715 10-001a: imx715_set_data_rate: Data rate: 4
[ 796.211730] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx715 10-001a bound
[ 796.212667] imx715 10-001a: Detected imx715 sensor
[ 796.212971] imx715 11-001a: initializing mipi...
[ 796.213414] imx715 11-001a: tegracam sensor driver:imx715_v2.0.6
[ 796.213588] imx715 11-001a: [S***] imx715_board_setup++
[ 796.213807] imx715 11-001a: imx715_power_on: power on
**[ 796.221201] imx715 10-001a: [S***] imx715_open:**
[ 796.245710] imx715 11-001a: imx715_calculate_line_time:++
[ 796.246451] imx715 11-001a: imx715_calculate_line_time: hmax: 550 [inck], INCK: 24000000 [Hz], line_time: 7407 [ns]
[ 796.246455] imx715 11-001a: imx715_power_off: power off
[ 796.246664] imx715 11-001a: imx715_set_operation_mode: set operation mode 0
[ 796.246675] imx715 11-001a: imx715_set_data_rate:++
[ 796.246678] imx715 11-001a: imx715_verify_data_rate++
[ 796.246680] imx715 11-001a: imx715_verify_data_rate: Selected data rate is not supported with 2 CSI lane mode, switching to default!
[ 796.246683] imx715 11-001a: imx715_set_data_rate: Data rate: 4
[ 796.246717] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx715 11-001a bound
[ 796.247292] imx715 11-001a: Detected imx715 sensor
**[ 796.250606] imx715 11-001a: [S***] imx715_open:**
[ 797.152963] imx715 11-001a: imx715_check_unsupported_mode++
[ 797.152978] imx715 11-001a: imx715_check_unsupported_mode++
[ 797.152982] imx715 11-001a: imx715_after_set_pixel_format++
[ 797.153096] imx715 11-001a: imx715_power_on: power on
[ 797.183694] imx715 11-001a: imx715_disable_second_slave_address: Sensors 2nd slave address disabled
[ 797.187191] imx715 11-001a: imx715_set_mode: set mode 0
[ 797.212937] imx715 11-001a: imx715_set_csi_lane_mode: sensor is in 2 CSI lane mode
[ 797.235181] imx715 11-001a: imx715_set_operation_mode: set operation mode 0
[ 797.235540] imx715 11-001a: imx715_configure_triggering_pins: Sensor is in - External sync Master mode
[ 797.235719] imx715 11-001a: imx715_configure_triggering_pins: XVS_XHS driver register: 3
[ 797.235723] imx715 11-001a: imx715_set_data_rate:++
[ 797.235726] imx715 11-001a: imx715_verify_data_rate++
[ 797.235731] imx715 11-001a: imx715_verify_data_rate: Selected data rate operates on clock frequency which is not supported, switching to default
[ 797.235733] imx715 11-001a: imx715_set_mclk:++
[ 797.236396] imx715 11-001a: imx715_set_mclk: enable MCLK with 24000000 Hz
[ 797.251203] imx715 11-001a: imx715_set_data_rate: Data rate: 4
[ 797.262690] imx715 11-001a: imx715_set_test_pattern++ Test mode pattern: 4294967295
[ 797.262697] imx715 11-001a: imx715_adjust_hmax_register:++
[ 797.263434] imx715 11-001a: imx715_adjust_hmax_register: HMAX: 1016
[ 797.263447] imx715 11-001a: imx715_calculate_line_time:++
[ 797.263814] imx715 11-001a: imx715_calculate_line_time: hmax: 1016 [inck], INCK: 24000000 [Hz], line_time: 14111 [ns]
[ 797.263819] imx715 11-001a: imx715_set_mode: set mode 0
[ 797.264549] imx715 11-001a: imx715_set_gain: gain val [0] reg [0]
[ 797.264558] imx715 11-001a: imx715_set_exposure: integration time: 33330 [us]
[ 797.265444] imx715 11-001a: imx715_set_exposure: set integration time: 33330 [us], coarse1:2362 [line], shr0: 2246 [line], frame length: 2250 [line]
[ 797.265450] imx715 11-001a: imx715_set_exposure: integration time: 33330 [us]
[ 797.266333] imx715 11-001a: imx715_set_exposure: set integration time: 33330 [us], coarse1:2362 [line], shr0: 262 [line], frame length: 2624 [line]
[ 797.267217] imx715 11-001a: imx715_set_frame_rate: val: 27000000, frame_length set: 2624
[ 797.267221] imx715 11-001a: imx715_set_frame_rate: frame rate: 27fps
[ 797.268350] imx715 11-001a: imx715_set_black_level: black level: 50
[ 797.268360] imx715 11-001a: [S***] imx715_start_streaming entered.
[ 801.846971] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx715 10-001a unbind
[ 801.847400] [S***] imx715 sensor i2c bus 10 removed
[ 802.062481] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx715 11-001a unbind
[ 802.062897] [S***] imx715 sensor i2c bus 11 removed
We can therefore see the two open imx715 ports (10-001a) & (11-001a), but only the 11-001a sensor continues to be initialized after the driver’s open() call.
On the syslog side, we do indeed find an argus-daemon error.
Sep 6 15:37:47 nano01 kernel: [ 797.268360] imx715 11-001a: [S***] imx715_start_streaming entered.
Sep 6 15:37:47 nano01 nvargus-daemon: Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/fusaCsiStreamHandler.cpp 248
Sep 6 15:37:47 nano01 nvargus-daemon: Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/fusaViHandler.cpp 328
Sep 6 15:37:47 nano01 nvargus-daemon: Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/fusaViHandler.cpp 124
Sep 6 15:37:47 nano01 nvargus-daemon: Module_id 30 Severity 2 : (fusa) Error: InvalidState propagating from:/capture/src/core.cpp 76
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/services/capture/FusaCaptureViCsiHw.cpp, function openViCsi(), line 393)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function setMode(), line 470)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/services/capture/CaptureRecord.cpp, function updateMode(), line 467)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1355)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/common/Utils.cpp, function workerThread(), line 114)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: Worker thread CaptureScheduler frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)
Sep 6 15:37:47 nano01 gnome-shell[1876]: Window manager warning: Buggy client sent a _NET_ACTIVE_WINDOW message with a timestamp of 0 for 0x2a00007
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error Timeout: (propagating from src/api/Buffer.cpp, function waitForUnlock(), line 644)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error Timeout: (propagating from src/components/CaptureContainerImpl.cpp, function returnBuffer(), line 426)
Sep 6 15:37:47 nano01 gnome-shell[1876]: Window manager warning: Buggy client sent a _NET_ACTIVE_WINDOW message with a timestamp of 0 for 0x2c00007
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: Capture Scheduler not running (in src/services/capture/CaptureServiceDevice.cpp, function addNewItemToSchedule(), line 1004)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/services/capture/CaptureService.cpp, function addRequest(), line 411)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/components/stages/SensorCaptureStage.cpp, function doHandleRequest(), line 87)
Sep 6 15:37:47 nano01 nvargus-daemon[18724]: SCF: Error InvalidState: (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
Any idea what the problem might be here?"
Thanks